用新方法自动发现对话模型隐藏错误,提升识别未知问题能力。
Towards Automated Error Discovery: A Study in Conversational AI
- 基于编码器设计新框架,通过对比学习增强错误表征
- 在多个数据集上检测未知错误准确率最高提升8个百分点
- 适合希望提升对话系统鲁棒性的研发团队使用
尽管基于大语言模型的对话代理表现出强流畅性与连贯性,但仍会产生难以预防的错误行为,尤其在部署后难以被发现。现有研究利用大语言模型检测错误并引导生成模型优化,但对未明确指令中的错误(如模型更新或用户行为变化引发的问题)仍难识别。本文提出自动化错误发现框架,并设计SEEED方法——一种基于编码器的误差检测方案。通过增强负样本的距离权重并引入基于标签的样本排序策略,提升表示学习效果。SEEED在多个标注错误的对话数据集上优于适配基线(包括GPT-4o和Phi-4),在未知错误检测上的准确率最高提升8个百分点,且在未知意图识别任务中展现强泛化能力。
原文摘要 · Abstract (English)
Although LLM-based conversational agents demonstrate strong fluency and coherence, they still produce undesirable behaviors (errors) that are challenging to prevent from reaching users during deployment. Recent research leverages large language models (LLMs) to detect errors and guide response-generation models toward improvement. However, current LLMs struggle to identify errors not explicitly specified in their instructions, such as those arising from updates to the response-generation model or shifts in user behavior. In this work, we introduce Automated Error Discovery, a framework for detecting and defining errors in conversational AI, and propose SEEED (Soft Clustering Extended Encoder-Based Error Detection), as an encoder-based approach to its implementation. We enhance the Soft Nearest Neighbor Loss by amplifying distance weighting for negative samples and introduce Label-Based Sample Ranking to select highly contrastive examples for better representation learning. SEEED outperforms adapted baselines -- including GPT-4o and Phi-4 -- across multiple error-annotated dialogue datasets, improving the accuracy for detecting unknown errors by up to 8 points and demonstrating strong generalization to unknown intent detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。