A Dual-branch Convolutional Network Architecture Processing on both Frequency and Time Domain for Single-channel Speech Enhancement

Zhang, Kanghao1; He, Shulin1; Li, Hao2; Zhang, Xueliang1

2023

10.1561/116.00000083

APSIPA TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING   Impact Factor & Quartile

2048-7703

Single-channel speech enhancement aims to remove the interfering noise and reverberation in real environments by a single microphone, which is a very challenging task in the speech signal processing field. Over the past years, deep learning has shown great potential for speech enhancement. In this paper, we propose a novel real-time framework, called DBCN, which is a dual-branch architecture. One branch takes waveform as its input for time-domain modeling and the other one takes shift real spectrum as input for frequency-domain modeling. The two branches have the same network structure, which is the representative convolutional recurrent network. To exchange information sufficiently, a bridge module is added between the two branches. Furthermore, we propose a novel feature normalization approach that enables each band to complete the normalization independently by counting the root mean square of each band and obtaining the inter-frame relationship for each band. The proposed approach allows the network to ignore the magnitude during processing, reducing learning difficulty and improving performance. Systematical evaluation and comparison are conducted. Experimental results show that the proposed system substantially outperforms related algorithms for causal and non-causal speech enhancement under very challenging environments.

Deep learning speech enhancement time-domain processing frequency-domain processing feature normalization

ESCI

English

Others

China National Nature Science Foundation["61876214","KF-2022-07-009"]

Engineering

Engineering, Electrical & Electronic

WOS:001030577500006

NOW PUBLISHERS INC

Web of Science

1.Inner Mongolia Univ, Coll Comp Sci, Hohhot, Peoples R China
2.Southern Univ Sci & Technol, Dept Elect & Elect Engn, Shenzhen, Peoples R China

Zhang, Kanghao,He, Shulin,Li, Hao,et al. A Dual-branch Convolutional Network Architecture Processing on both Frequency and Time Domain for Single-channel Speech Enhancement[J]. APSIPA TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING,2023,12(3).

Zhang, Kanghao,He, Shulin,Li, Hao,&Zhang, Xueliang.(2023).A Dual-branch Convolutional Network Architecture Processing on both Frequency and Time Domain for Single-channel Speech Enhancement.APSIPA TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING,12(3).

Zhang, Kanghao,et al."A Dual-branch Convolutional Network Architecture Processing on both Frequency and Time Domain for Single-channel Speech Enhancement".APSIPA TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING 12.3(2023).