arXiv:2509.04871cs.AIcs.LG2025-09被引 2

从电话录音克隆销售语音助手,实现实时对话与剧本执行。

Cloning a Conversational Voice AI Agent from Call\,Recording Datasets for Telesales

  • 基于通话记录训练,整合语音识别与大模型实现端到端对话。
  • 盲测显示在流程执行上接近真人,但在说服力和异议处理上仍有差距。
  • 适合需要自动化销售话术的场景,如客服、电销等应用落地。

近期语言与语音建模的进步使得构建能实时理解与生成人类对话的自主语音助手成为可能。这类系统正被广泛应用于客户服务与医疗等领域,可自动化重复任务、降低运营成本并提供全天候支持。本文提出一种通用方法,从电话录音语料库中克隆对话式语音AI代理。虽然以电销数据为例展示该方法,但其流程可推广至任何拥有通话转录数据的领域。系统通过电话倾听客户,以合成语音回应,并依据顶尖人工代理的学习结果遵循结构化话术剧本。我们介绍了领域选择、知识提取与提示工程,将自动语音识别、基于大语言模型的对话管理器与文本转语音合成集成到流式推理管道中。该克隆代理在22项标准(涵盖开场、产品介绍、销售推动、异议处理与成交)上与真人代理对比评估。盲测结果显示,该AI代理在通话常规环节表现接近人类,但在说服力与异议处理方面仍显不足。我们分析了这些短板并优化提示策略。论文最后总结设计经验与未来研究方向,包括大规模仿真与自动化评估。

原文摘要 · Abstract (English)

Recent advances in language and speech modelling have made it possible to build autonomous voice assistants that understand and generate human dialogue in real time. These systems are increasingly being deployed in domains such as customer service and healthcare care, where they can automate repetitive tasks, reduce operational costs, and provide constant support around the clock. In this paper, we present a general methodology for cloning a conversational voice AI agent from a corpus of call recordings. Although the case study described in this paper uses telesales data to illustrate the approach, the underlying process generalizes to any domain where call transcripts are available. Our system listens to customers over the telephone, responds with a synthetic voice, and follows a structured playbook learned from top performing human agents. We describe the domain selection, knowledge extraction, and prompt engineering used to construct the agent, integrating automatic speech recognition, a large language model based dialogue manager, and text to speech synthesis into a streaming inference pipeline. The cloned agent is evaluated against human agents on a rubric of 22 criteria covering introduction, product communication, sales drive, objection handling, and closing. Blind tests show that the AI agent approaches human performance in routine aspects of the call while underperforming in persuasion and objection handling. We analyze these shortcomings and refine the prompt accordingly. The paper concludes with design lessons and avenues for future research, including large scale simulation and automated evaluation.

语音助手对话系统电销自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。