A Study of Modeling Rising Intonation in Cantonese Neural Speech Synthesis

Bai，Qibing1; Ko，Tom2; Zhang，Yu1

10.21437/Interspeech.2022-11173

2022

Interspeech Conference

2308-457X

1990-9772

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

2022-September

501-505

SEP 18-22, 2022

null,Incheon,SOUTH KOREA

C/O EMMANUELLE FOXONET, 4 RUE DES FAUVETTES, LIEU DIT LOUS TOURILS, BAIXAS, F-66390, FRANCE

ISCA-INT SPEECH COMMUNICATION ASSOC

In human speech, the attitude of a speaker cannot be fully expressed only by the textual content. It has to come along with the intonation. Declarative questions are commonly used in daily Cantonese conversations, and they are usually uttered with rising intonation. Vanilla neural text-to-speech (TTS) systems are not capable of synthesizing rising intonation for these sentences due to the loss of semantic information. Though it has become more common to complement the systems with extra language models, their performance in modeling rising intonation is not well studied. In this paper, we propose to complement the Cantonese TTS model with a BERT-based statement/question classifier. We design different training strategies and compare their performance. We conduct our experiments on a Cantonese corpus named CanTTS. Empirical results show that the separate training approach obtains the best generalization performance and feasibility.

declarative questions intonation prosody text-to-speech

First

English

CPCI-S

Acoustics ; Audiology & Speech-Language Pathology ; Computer Science ; Engineering

Acoustics ; Audiology & Speech-Language Pathology ; Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic

WOS:000900724500102

2-s2.0-85140073777

Scopus

1.Department of Computer Science and Engineering,Southern University of Science and Technology,China
2.ByteDance AI Lab,

Bai，Qibing,Ko，Tom,Zhang，Yu. A Study of Modeling Rising Intonation in Cantonese Neural Speech Synthesis[C]. C/O EMMANUELLE FOXONET, 4 RUE DES FAUVETTES, LIEU DIT LOUS TOURILS, BAIXAS, F-66390, FRANCE:ISCA-INT SPEECH COMMUNICATION ASSOC,2022:501-505.