论文标题
统一的单枪韵律和演讲者转换系统，具有自我监督的离散语音单位
A unified one-shot prosody and speaker conversion system with self-supervised discrete speech units
论文作者
论文摘要
我们提出了一个统一的系统，可以在音调，节奏和扬声器属性上实现单发语音转换（VC）。现有作品通常忽略了韵律与语言内容之间的相关性，从而导致converted依的语音中的自然性退化。此外，缺乏适当的语言功能可防止这些系统在转换后准确保留语言内容。为了解决这些问题，我们设计了一个级联的模块化系统，利用自我监督的离散语音单元作为语言表示。这些离散单元为节奏建模提供了必不可少的持续时间信息。我们的系统首先从原始波形中提取了话语级韵律和说话者表示。鉴于韵律表示，韵律预测器估计了话语中每个离散单元的音高，能量和持续时间。合成器进一步根据预测的韵律，说话者表示和离散单位重建语音。实验表明，我们的系统在自然性，清晰度，说话者的可转移性和韵律转移性方面都优于先前的方法。代码和样本公开可用。
We present a unified system to realize one-shot voice conversion (VC) on the pitch, rhythm, and speaker attributes. Existing works generally ignore the correlation between prosody and language content, leading to the degradation of naturalness in converted speech. Additionally, the lack of proper language features prevents these systems from accurately preserving language content after conversion. To address these issues, we devise a cascaded modular system leveraging self-supervised discrete speech units as language representation. These discrete units provide duration information essential for rhythm modeling. Our system first extracts utterance-level prosody and speaker representations from the raw waveform. Given the prosody representation, a prosody predictor estimates pitch, energy, and duration for each discrete unit in the utterance. A synthesizer further reconstructs speech based on the predicted prosody, speaker representation, and discrete units. Experiments show that our system outperforms previous approaches in naturalness, intelligibility, speaker transferability, and prosody transferability. Code and samples are publicly available.
