Abstract:
Single acoustic features are insufficient to fully convey vocal emotions, and simple concatenation methods of fusing multiple acoustic features can lead to loss of emotional information, while single-stream feature extraction networks are inadequate for comprehensively extracting emotional features from speech. To address these issues, this paper proposes a dual- stream feature fusion network for speech emotion recognition. The system inputs four acoustic features (Mel-frequency cepstral coefficients, root mean square, zero-crossing rate, and chroma short-time Fourier transform) that contribute significantly to the distinction of emotional categories. It employs a dual-stream network architecture to extract both short-term local features and global contextual features. The network integrates a local feature extraction network based on one-dimensional dilated convolutions and a global feature extraction network utilizing a bi-directional long short-term memory network with a self-attention mechanism in parallel, thus preventing interference of information. An ensemble learning method with a voting strategy is utilized to achieve comprehensive fusion of deep features from each stream, extracting more profound emotional information and improving classification accuracy. Experimental results demonstrate that the dual-stream multi-feature fusion network model achieves emotion recognition accuracies of 99.89%, 95.89%, 96.61%, 97.75%, and 95.13% on the TESS, RAVDESS, SAVEE, CREMA-D datasets, and a mixed dataset, respectively. Compared to other similar speech emotion recognition models, it exhibits superior performance, with recognition accuracies surpassing those of alternative models.