用提示词让大模型在数据库出错时自动纠错,避免胡编乱造。
When the Database Fails: Prompting LLM Dialogue Agents for Safe Recovery in Task-Oriented Dialogue

- 设计提示词引导模型根据数据库状态修复错误响应
- 减少50%以上幻觉,无需重新训练或额外调用
- 适合做客服、预订类对话系统的开发者参考
在任务型对话中,大语言模型在后端数据库调用失败、返回空结果或信息不匹配时,常生成看似流畅但不安全的回应,如虚构场所、确认信息或预订详情。本文提出一种轻量级提示词恢复方法,无需重训练或额外模型调用即可提升鲁棒性。在六种开源模型(DeepSeek-R1、Gemma-2、Llama-3、Mistral、Phi-3、Qwen-2.5)和四种数据库场景(空结果、错域检索、API错误、正常检索)下测试,使用基于MultiWOZ 2.2(5个领域)和SGD(20个领域)构建的故障注入基准。结果显示,原始模型在MultiWOZ上30.5%、SGD上20.9%的故障回合出现幻觉;采用引导重试策略后,分别降至15.3%和12.2%,降幅达50%和42%。但残余幻觉仍显著(各模型6%-37%),错域失败最难处理。结果在两数据集及所有模型上一致,人工标注验证了自动承诺安全性度量的有效性。
原文摘要 · Abstract (English)
Large language models used in task-oriented dialogue often produce fluent but unsafe responses when backend database calls fail, return empty results, or surface mismatched information, inventing venues, confirmations, or booking details not grounded in the database. We study a lightweight prompting-based recovery approach that improves robustness without retraining or additional model calls. We compare three response strategies, including a guided recovery prompt conditioned on structured database status, across six open-weight model families (DeepSeek-R1, Gemma-2, Llama-3, Mistral, Phi-3, and Qwen-2.5) and four database conditions: empty result, wrong-domain retrieval, API error, and clean retrieval. Using fault-injected benchmarks built on two structurally different datasets, MultiWOZ 2.2 (5 domains) and SGD (20 domains), we find that naive agents hallucinate on 30.5% of failure turns on MultiWOZ and 20.9% on SGD. Our Guided-Retry strategy reduces hallucination by 50% on MultiWOZ (30.5 to 15.3%) and by 42% on SGD (20.9 to 12.2%) without retraining. However, residual hallucination remains substantial (6-37% across models), with wrong-domain failures the hardest case. Results are consistent across both datasets and all six model families, and human annotation shows substantial agreement while supporting the validity of the automatic commitment-safety metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。