语音克隆实为风格迁移，会改变原声特征并影响信任感
Voice "Cloning" is Style Transfer
- 发现主流语音克隆模型本质是风格迁移而非真实复制
- 克隆语音被评价更权威、更温暖，可信度更高
- 适合关注语音合成伦理与用户信任的研究者阅读
生成式语音正广泛应用于日常生活。语音克隆技术尤其适用于需保留身份特征的场景，如补录音频、多语言配音或帮助失语者保留声音。然而，本研究发现，尽管称为“克隆”，实际并未忠实复现个体声音。人类评估显示，克隆语音相比源语音更具权威性、温暖感、客服风格及人声质感。受试者对克隆语音的信任度更高，更愿意向其透露敏感信息。此外，语音克隆导致说话人特征同质化，表现为口音、语速和音频嵌入空间方差下降。这些结果揭示了语音克隆技术的新局限与潜在风险及其对人类行为的影响。
原文摘要 · Abstract (English)
Artificially generated speech is increasingly embedded in everyday life. Voice cloning in particular enables applications where identity preservation is important, such as completing a recording, dubbing in a new language, or preserving the voices of individuals with speech loss. However, in our work, we find that despite the term, voice cloning does not faithfully ''clone'' an individual's voice. Instead, we find that widely-used voice cloning models systematically apply style transfer to source voices. As rated by human annotators, cloned voices are perceived as more authoritative, warm, customer-service-like, and human-like compared to their sources. Human annotators also report greater trust in cloned voices than source voices, and a greater willingness to disclose sensitive personal information to them. Our work furthermore shows that voice cloning leads to homogenization of speaker characteristics, as measured by reduced variance in accent, speaking rate, and the audio embedding space. Together, our results highlight a new set of limitations and risks of voice cloning technology and their potential impact on human behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。