arXiv:2607.17900cs.SD2026-07

用轻量层让语音合成更懂上下文,实时响应更自然。

Harness TTS: Towards Context-Aware Expressive Speech Synthesis with Harness Layer

  • 引入可插拔控制层,通过提示工具路由实现风格调控。
  • 在多种场景下指令遵循准确率提升23.1至35.6个百分点。
  • 4B模型推荐延迟低于50毫秒,适合实时语音助手使用。

语音助手的自然表达需要灵活适配显式请求与交互上下文的风格控制。本文提出Harness TTS,一种轻量级控制层,嵌入于TTS引擎外,用于外部化并管理其表达行为。该方法将风格控制重构为闭集提示工具路由:离线构建含结构化元数据的紧凑风格提示工具库;在线由大模型规划器根据优先级感知观察模式选择合适工具,由TTS执行器使用对应提示音频生成语音。在路由与合成任务上评估显示,Qwen3-4B在显式、隐式及冲突子集上的Top-1准确率分别为74.3%、43.0%和64.6%。在合成方面,于CosyVoice3和VoxCPM2上,Harness TTS相比仅用指令控制,指令遵循胜率提升23.1–35.6点(CosyVoice3)与13.8–20.0点(VoxCPM2),UTMOSv2评分提升0.11–0.38。此外,4B规划器在标准模式下首次推荐延迟低于50毫秒,对实时交互几乎无额外延迟。结果表明,为TTS引擎配备专用Harness层,是一种实用、可审计且上下文感知的语音表达控制方案。

原文摘要 · Abstract (English)

Expressive speech synthesis for voice assistants requires flexible style control that adapts to explicit requests and broader interaction context. We propose Harness TTS, a lightweight control layer that wraps around a TTS engine to externalize and govern its expressive behavior. It reformulates style control as closed-set prompt-tool routing: offline, a compact registry of stylistic prompt tools is constructed with structured metadata; online, an LLM planner selects the appropriate tool based on a priority-aware observation schema, and the TTS executor synthesizes speech using the corresponding prompt audio. We evaluate Harness TTS on both routing and synthesis tasks. In routing, Qwen3-4B achieves Top-1 accuracies of 74.3%, 43.0%, and 64.6% on explicit, implicit, and conflict subsets. For synthesis, experiments on CosyVoice3 and VoxCPM2 show that Harness TTS outperforms instruction-only control, achieving higher instruction-following win rates (margins of 23.1-35.6 points on CosyVoice3 and 13.8-20.0 points on VoxCPM2) and improving UTMOSv2 scores by 0.11-0.38. Moreover, the 4B planner delivers its first tool recommendation in under 50 ms in standard mode, introducing negligible latency for real-time interaction. These results demonstrate that equipping TTS engines with a dedicated Harness layer offers a practical, auditable, and context-aware solution for voice assistant expression control.

语音合成风格控制上下文感知轻量层

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。