单文件框架实现主动对话，让语音交互更像真人。
CleanS2S: Single-file Framework for Proactive Speech-to-Speech Interaction
- 整合语音识别、大模型、语音合成，支持实时打断
- 提出五种类人应答策略，响应延迟低且可主动发起对话
- 代码开源，单文件结构便于研究者快速部署与扩展
CleanS2S 是一个面向类人语音对话的单文件框架，通过集成自动语音识别、大语言模型和文本转语音合成，构建统一实时交互管道。系统采用全双工 WebSocket 和非阻塞 I/O 实现低延迟切换，支持实时中断。突破传统回合制模式，引入主动交互机制：结合记忆模块与主观动作判断模块，实现中断、拒绝、转移、沉默和标准回复五种类人响应策略。记忆模块动态聚合历史与上下文信息以支持决策。我们还提出动作判断监督微调（Action Judgement SFT），用于评估输入流并选择响应策略。框架采用原子化配置，单文件实现，提供前所未有的透明性与可扩展性。代码已开源：\https://github.com/opendilab/CleanS2S。
原文摘要 · Abstract (English)
CleanS2S is a framework for human-like speech-to-speech interaction that advances conversational AI through single-file implementation and proactive dialogue capabilities. Our system integrates automatic speech recognition, large language models, and text-to-speech synthesis into a unified pipeline with real-time interruption handling, achieving low transition latency through full-duplex websocket connections and non-blocking I/O. Beyond conventional chatbot paradigms, we pioneer a proactive interaction mechanism, which combines memory systems with Subjective Action Judgement module, enabling five human-like response strategies: interruption, refusal, deflection, silence, and standard response. The memory module dynamically aggregates historical, and contextual data to inform interaction decisions. This approach breaks the rigid turn-based convention by allowing system-initiated dialog control and context-aware response selection. And we propose Action Judgement SFT that assesses input streams for responses strategies. The framework's single-file implementation with atomic configurations offers researchers unprecedented transparency and extensibility for interaction agents. The code of CleanS2S is released at \https://github.com/opendilab/CleanS2S.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。