arXiv:2507.15221cs.SDeess.AS2025-07被引 1

为老人和孩子打造可延续的数字人格,留住声音与记忆。

EchoVoices: Preserving Generational Voices and Memories for Seniors and Children

  • 用k-NN增强Whisper模型识别老人儿童的特殊语音
  • 通过年龄自适应的VITS模型生成高保真语音
  • 结合LLM与RAG实现对话连贯,适合家庭传承

近期智能语音与数字人技术多聚焦成年主流用户,忽视了老人与儿童独特的语音特征和交互方式。这些群体在发音、语言风格和互动模式上显著区别于常规系统,挑战现有自动语音识别(ASR)、文本转语音(TTS)和大语言模型(LLM)的能力。为此,我们提出EchoVoices——一个端到端的数字人系统,专为老人与儿童创建持久的数字人格,确保其声音与记忆得以传承。系统集成三项核心创新:基于k-NN增强的Whisper模型,提升异常语音的识别鲁棒性;年龄自适应的VITS模型,实现高保真、说话人感知的语音合成;以及由LLM驱动的智能体,自动构建人物卡,并利用基于检索增强生成(RAG)的记忆系统维持对话一致性。在SeniorTalk与ChildMandarin数据集上的实验表明,该系统在识别准确率、合成质量与说话人相似度方面均有显著提升。EchoVoices提供了一个完整的代际声音保存框架,为跨代情感连接与数字遗产创造开辟新路径。

原文摘要 · Abstract (English)

Recent breakthroughs in intelligent speech and digital human technologies have primarily targeted mainstream adult users, often overlooking the distinct vocal patterns and interaction styles of seniors and children. These demographics possess distinct vocal characteristics, linguistic styles, and interaction patterns that challenge conventional ASR, TTS, and LLM systems. To address this, we introduce EchoVoices, an end-to-end digital human pipeline dedicated to creating persistent digital personas for seniors and children, ensuring their voices and memories are preserved for future generations. Our system integrates three core innovations: a k-NN-enhanced Whisper model for robust speech recognition of atypical speech; an age-adaptive VITS model for high-fidelity, speaker-aware speech synthesis; and an LLM-driven agent that automatically generates persona cards and leverages a RAG-based memory system for conversational consistency. Our experiments, conducted on the SeniorTalk and ChildMandarin datasets, demonstrate significant improvements in recognition accuracy, synthesis quality, and speaker similarity. EchoVoices provides a comprehensive framework for preserving generational voices, offering a new means of intergenerational connection and the creation of lasting digital legacies.

数字人语音合成代际传承老年人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。