将对话助手从单一模型改为分步协作,提升准确率与效率。
From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale
- 用分步工具调用替代单模型处理,按角色分工执行任务。
- 精准度提升至89.1%,结构化幻觉减少至0.0%,错误升级率下降超40%。
- 适合高并发客服系统优化,尤其关注成本与响应质量的团队。
大型住宿平台的客服助手每月处理数百万次对话,支持11种语言,90%响应延迟需在10秒内。本文提出动态响应(DR)架构,取代原有的单模型融合系统(Qwen3-235B-A22B),改用带类型约束的ReAct编排器搭配小型生成器,依据后端验证的上下文契约进行输出。通过严格对照实验,确认架构改进效果:类型化实体选择使预订选择器精度从8.3%提升至89.1%,召回率从75.2%降至67.3%;类型化动作ID结合成员校验,彻底消除结构化动作幻觉(从2.14%降至0.0%)。低速上线的A/B测试重现了回放结果:硬升级率由5.60%降至3.08%,软升级率由9.56%降至2.49%,人工转接量基本稳定;自解决率提升5.1点(95%置信区间[-2, +12])。服务优化使编排器P90延迟从3.87秒降至2.24秒,GPU占用减少约三分之一,自托管使年模型服务成本降低一个数量级以上。
原文摘要 · Abstract (English)
Conversational assistants can blend retrieval, action selection, escalation, and wording in a single model path, or separate those roles. We report a production migration of a customer-support assistant at a large accommodation marketplace (millions of conversations per month, 11 languages, 10-second P90). Dynamic Response (DR) replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools plus a smaller generator that writes from a backend-validated context contract. Because the migration also changed prompts, alignment, and serving, we attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns: typed entity selection moves the reservation selector to a precision-first operating point (precision 8.3% to 89.1%, recall 75.2% to 67.3%), and typed action IDs with a membership check remove observed structured-action hallucination (2.14% to 0.0%). A low-ramp A/B test reproduces the replay escalation reductions: hard-escalation responses fall from 5.60% to 3.08% and soft-escalation responses from 9.56% to 2.49%, while production handoff volume holds roughly steady; self-solve is directional (+5.1 points, 95% CI [-2, +12]). Serving optimizations cut orchestrator P90 latency from 3.87s to 2.24s on a GPU footprint reduced by roughly one-third, and self-hosting reduces estimated annual model-serving cost by more than an order of magnitude.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。