让AI说话更像人，通过插入口误提升自然感。
Enhancing Naturalness in LLM-Generated Utterances through Disfluency Insertion
- 用低秩适配微调LLM，加入口误等自然语言特征。
- 用户实验显示口误使语音感知更自然，但稍降低清晰度。
- 适合开发拟真人对话系统的研究人员或工程师。
口误是人类自发言语的自然特征，但大型语言模型（LLMs）生成的语句通常缺乏此类特征，导致合成语音的自然感下降，影响对话系统的人类化表现。本文提出一种方法：首先使用低秩适配（LoRA）对LLM进行微调，使其在生成语句中引入多种口误类型；随后利用支持口误生成的文本转语音模型合成语音。通过两个指标评估生成语音质量：可理解性和感知自发性。用户研究表明，插入口误显著提升了语音的感知自发性，但伴随轻微的可理解性下降。
原文摘要 · Abstract (English)
Disfluencies are a natural feature of spontaneous human speech but are typically absent from the outputs of Large Language Models (LLMs). This absence can diminish the perceived naturalness of synthesized speech, which is an important criteria when building conversational agents that aim to mimick human behaviours. We show how the insertion of disfluencies can alleviate this shortcoming. The proposed approach involves (1) fine-tuning an LLM with Low-Rank Adaptation (LoRA) to incorporate various types of disfluencies into LLM-generated utterances and (2) synthesizing those utterances using a text-to-speech model that supports the generation of speech phenomena such as disfluencies. We evaluated the quality of the generated speech across two metrics: intelligibility and perceived spontaneity. We demonstrate through a user study that the insertion of disfluencies significantly increase the perceived spontaneity of the generated speech. This increase came, however, along with a slight reduction in intelligibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。