Multi-View Self-Attention Based Transformer for Speaker Recognition

Rui Wang1; Junyi Ao2; Long Zhou3; Shujie Liu3; Zhihua Wei1; Tom Ko2; Qing Li4; Yu Zhang2

10.1109/ICASSP43922.2022.9746639

2022

47th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

1520-6149

978-1-6654-0541-6

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

2022-May

6732-6736

23-27 May 2022

Singapore, Singapore

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Initially developed for natural language processing (NLP), Transformer model is now widely used for speech processing tasks such as speaker recognition, due to its powerful sequence modeling capabilities. However, conventional self-attention mechanisms are originally designed for modeling textual sequence without considering the characteristics of speech and speaker modeling. Besides, different Transformer variants for speaker recognition have not been well studied. In this work, we propose a novel multi-view self-attention mechanism and present an empirical study of different Transformer variants with or without the proposed attention mechanism for speaker recognition. Specifically, to balance the capabilities of capturing global dependencies and modeling the locality, we propose a multi-view self-attention mechanism for speaker Transformer, in which different attention heads can attend to different ranges of the receptive field. Furthermore, we introduce and compare five Transformer variants with different network architectures, embedding locations, and pooling methods to learn speaker embeddings. Experimental results on the VoxCeleb1 and VoxCeleb2 datasets show that the proposed multi-view self-attention mechanism achieves improvement in the performance of speaker recognition, and the proposed speaker Transformer network attains excellent results compared with state-of-the-art models.

speaker recognition Transformer speaker identification speaker verification

Others

English

EI ; CPCI-S

National Nature Science Foundation of China["61976160","62076182","61906137"] ; Technology research plan project of Ministry of Public and Security[2020JSYJD01] ; Shanghai Science and Technology Plan Project[21DZ1204800]

Acoustics ; Computer Science ; Engineering

Acoustics ; Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic

WOS:000864187907007

20222312199281

IEEE

1.Tongji University,Department of Computer Science and Technology
2.Southern University of Science and Technology,Department of Computer Science and Engineering
3.Microsoft Research Asia
4.The Hong Kong Polytechnic University,Department of Computing

Rui Wang,Junyi Ao,Long Zhou,et al. Multi-View Self-Attention Based Transformer for Speaker Recognition[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2022:6732-6736.