让大模型学会答错时主动放弃，提升可靠性和安全性。
Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMs
- 通过逆向推理轨迹重构原始问题，判断模型是否答偏了。
- 在9个数据集上显著提升4个前沿模型的拒答能力，36次测试中33次胜过基线。
- 适合需要高可靠性、避免错误回答的场景，如医疗和法律决策。
为使大语言模型（LLMs）可靠部署，模型必须有效识别何时不应回答：即选择拒答。尽管推理模型在复杂任务上表现优异，但其拒答能力较差。针对这一缺陷，我们提出查询错位框架（Query Misalignment Framework）。将导致错误拒答的幻觉重新理解为模型回答了错误的问题，而非错误地回答了问题。基于此框架，我们开发了一种新型最先进的拒答方法——推理轨迹逆向（Trace Inversion）。首先生成模型的推理轨迹；然后仅依据该轨迹，重构模型最可能响应的原始问题；最后比较初始问题与重构问题。若两者相似度低，则表明模型很可能答错了，应被标记为拒答。大量实验表明，该方法在四个前沿大模型上，于九个拒答问答数据集上均显著提升拒答性能，在36组对比中，有33组优于现有基线。
原文摘要 · Abstract (English)
For Large Language Models (LLMs) to be reliably deployed, models must effectively know when not to answer: abstain. Reasoning models, in particular, have gained attention for impressive performance on complex tasks. However, reasoning models have been shown to have worse abstention abilities. Taking the vulnerabilities of reasoning models into account, we propose our Query Misalignment Framework. Hallucinations resulting in failed abstention can be reinterpreted as LLMs answering the wrong question (rather than answering a question incorrectly). Based on this framework, we develop a new class of state-of-the-art abstention methods called Trace Inversion. First, we generate the reasoning trace of a model. Based on only the trace, we then reconstruct the most likely query that the model responded to. Finally, we compare the initial query with the reconstructed query. Low similarity score between the initial query and reconstructed query suggests that the model likely answered the question incorrectly and is flagged to abstain. Extensive experiments demonstrate that Trace Inversion effectively boosts abstention performance in four frontier LLMs across nine abstention QA datasets, beating competitive baselines in 33 out of 36 settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。