Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation

Dong，Qianqian1; Yue，Fengpeng2; Ko，Tom1; Wang，Mingxuan1; Bai，Qibing2; Zhang，Yu2,3

10.21437/Interspeech.2022-10011

2022

Interspeech Conference

2308-457X

1990-9772

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

2022-September

1781-1785

SEP 18-22, 2022

null,Incheon,SOUTH KOREA

C/O EMMANUELLE FOXONET, 4 RUE DES FAUVETTES, LIEU DIT LOUS TOURILS, BAIXAS, F-66390, FRANCE

ISCA-INT SPEECH COMMUNICATION ASSOC

Direct Speech-to-speech translation (S2ST) has drawn more and more attention recently. The task is very challenging due to data scarcity and complex speech-to-speech mapping. In this paper, we report our recent achievements in S2ST. Firstly, we build a S2ST Transformer baseline which outperforms the original Translatotron. Secondly, we utilize the external data by pseudo-labeling and obtain a new state-of-the-art result on the Fisher English-to-Spanish test set. Indeed, we exploit the pseudo data with a combination of popular techniques which are not trivial when applied to S2ST. Moreover, we evaluate our approach on both syntactically similar (Spanish-English) and distant (English-Chinese) language pairs. Our implementation is available at https://github.com/fengpeng-yue/speech-to-speech-translation.

pseudo-labeling speech translation speech-to-speech translation

Others

English

CPCI-S

Acoustics ; Audiology & Speech-Language Pathology ; Computer Science ; Engineering

Acoustics ; Audiology & Speech-Language Pathology ; Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic

WOS:000900724501193

2-s2.0-85140060672

Scopus

1.ByteDance AI Lab,
2.Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,China
3.Peng Cheng Laboratory,Shenzhen,China

Dong，Qianqian,Yue，Fengpeng,Ko，Tom,et al. Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation[C]. C/O EMMANUELLE FOXONET, 4 RUE DES FAUVETTES, LIEU DIT LOUS TOURILS, BAIXAS, F-66390, FRANCE:ISCA-INT SPEECH COMMUNICATION ASSOC,2022:1781-1785.