MTI-Net: A Multi-Target Speech Intelligibility Prediction Model

Zezario，Ryandhimas E.1,2; Fu，Szu Wei3; Chen，Fei4; Fuh，Chiou Shann1; Wang，Hsin Min2; Tsao，Yu2

10.21437/Interspeech.2022-10828

2022

Interspeech Conference

2308-457X

1990-9772

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

2022-September

5463-5467

SEP 18-22, 2022

null,Incheon,SOUTH KOREA

C/O EMMANUELLE FOXONET, 4 RUE DES FAUVETTES, LIEU DIT LOUS TOURILS, BAIXAS, F-66390, FRANCE

ISCA-INT SPEECH COMMUNICATION ASSOC

Recently, deep learning (DL)-based non-intrusive speech assessment models have attracted great attention. Many studies report that these DL-based models yield satisfactory assessment performance and good flexibility, but their performance in unseen environments remains a challenge. Furthermore, compared to quality scores, fewer studies elaborate deep learning models to estimate intelligibility scores. This study proposes a multi-task speech intelligibility prediction model, called MTI-Net, for simultaneously predicting human and machine intelligibility measures. Specifically, given a speech utterance, MTI-Net is designed to predict human subjective listening test results and word error rate (WER) scores. We also investigate several methods that can improve the prediction performance of MTI-Net. First, we compare different features (including low-level features and embeddings from self-supervised learning (SSL) models) and prediction targets of MTI-Net. Second, we explore the effect of transfer learning and multi-tasking learning on training MTI-Net. Finally, we examine the potential advantages of fine-tuning SSL embeddings. Experimental results demonstrate the effectiveness of using cross-domain features, multi-task learning, and fine-tuning SSL embeddings. Furthermore, it is confirmed that the intelligibility and WER scores predicted by MTI-Net are highly correlated with the ground-truth scores.

self-supervised learning speech intelligibility prediction STOI Subjective listening tests WER

Others

English

CPCI-S

Acoustics ; Audiology & Speech-Language Pathology ; Computer Science ; Engineering

Acoustics ; Audiology & Speech-Language Pathology ; Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic

WOS:000900724505130

2-s2.0-85140047138

Scopus

1.National Taiwan University,Taiwan
2.Academia Sinica,
3.Microsoft Corporation,
4.Southern University of Science and Technology of China,China

Zezario，Ryandhimas E.,Fu，Szu Wei,Chen，Fei,et al. MTI-Net: A Multi-Target Speech Intelligibility Prediction Model[C]. C/O EMMANUELLE FOXONET, 4 RUE DES FAUVETTES, LIEU DIT LOUS TOURILS, BAIXAS, F-66390, FRANCE:ISCA-INT SPEECH COMMUNICATION ASSOC,2022:5463-5467.