论文标题
基于元学习的多语言语音合成中的发音和韵律建模
Decoupled Pronunciation and Prosody Modeling in Meta-Learning-Based Multilingual Speech Synthesis
论文作者
论文摘要
本文提出了一种解耦发音和韵律建模的方法，以提高基于元学习的多语言语音综合的性能。基线元学习合成方法采用了一个单个文本编码器，该文本编码器具有参数生成器，该参数生成器以语言嵌入和单个解码器进行预测，以预测所有语言的MEL光谱图。相比之下，我们提出的方法设计了一个两流模型结构，其中包含两个编码器和两个用于发音和韵律建模的解码器，考虑到发音知识和韵律知识应在语言之间以不同的方式共享。在我们的实验中，我们提出的方法有效地提高了与基线元学习合成方法相比，多语言语音合成的可理解性和自然性。
This paper presents a method of decoupled pronunciation and prosody modeling to improve the performance of meta-learning-based multilingual speech synthesis. The baseline meta-learning synthesis method adopts a single text encoder with a parameter generator conditioned on language embeddings and a single decoder to predict mel-spectrograms for all languages. In contrast, our proposed method designs a two-stream model structure that contains two encoders and two decoders for pronunciation and prosody modeling, respectively, considering that the pronunciation knowledge and the prosody knowledge should be shared in different ways among languages. In our experiments, our proposed method effectively improved the intelligibility and naturalness of multilingual speech synthesis comparing with the baseline meta-learning synthesis method.
