Dual-stream spatiotemporal graph convolutional networks for EEG-based human emotion recognition

Abstract

Deep learning has advanced EEG-based human emotion recognition, yet most existing approaches rely on either temporal or spectral features and insufficiently model the fine-grained spatiotemporal structure of neural activity. To address these challenges, this paper develops a dual-stream spatiotemporal graph convolutional network (DSSGCN) for human emotion recognition. In the time domain, a multi-scale modern temporal convolutional network (MS-MTCN) is designed to capture rich temporal information across diverse receptive fields and model long-range temporal dependencies. In the frequency domain, a fully-connected multi-scale graph attention network (FM-GAT) is introduced to learn complex inter-channel relationships and spatial dependencies from the spectral representation of EEG signals. Furthermore, a cross-domain feature fusion module (CFFM) is employed to integrate the complementary information from both temporal and spectral branches, followed by an adaptive ensemble classifier (AEC) to enhance recognition robustness. Finally, an improved online knowledge distillation (IOKD) algorithm is devised to enhance the model's robustness and generalization. Evaluated on two public dataset and a self-collected music-emotion dataset, DSSGCN achieves 93.98%, 85.00%, and 99.20% accuracy, consistently surpassing eleven state-of-the-art methods and validating its effectiveness for decoding affective states from EEG signals.