让大模型学会识别并放弃无法解决的问题，避免胡言乱语误导用户。
Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning
- 通过奖励机制引导模型在超出能力时拒绝推理，实现行为与能力对齐。
- 在复杂任务上减少87%的无效推理，同时保持原有任务性能不变。
- 适合需要高可靠性、防误导的AI应用，如医疗、金融决策系统。
大型语言模型在超出自身能力的任务上会产生计算开销大但语义无效的推理，导致看似合理却错误的推导误导用户。本文系统分析了这一‘无效推理’现象，揭示了能力过载和行为不匹配的普遍问题。主要失败模式为伪合理性推理：表面合法但存在细微错误，且随任务难度上升而加剧。为此，提出CaRL（能力对齐强化学习）框架，通过奖励塑形激励模型在无力推理时拒绝，以及事后拒绝增强将失败案例转化为拒绝监督信号。实验表明，该方法显著降低无效推理（减少87%），同时在各类任务中保持原有性能，有效实现能力边界对齐而不牺牲实用性。
原文摘要 · Abstract (English)
Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks, creating risks where plausible-sounding but incorrect derivations mislead users. We characterize this \textit{futile reasoning} phenomenon through systematic analysis, revealing universal capability overreach and systematic miscalibration between capability and behavior. The dominant failure mode is specious reasoning, which outputs look superficially valid but contain subtle errors, escalating with task difficulty. To address this, we introduce \textbf{CaRL} (\textbf{Ca}pability-\textbf{a}ligned \textbf{R}einforcement \textbf{L}earning), which aligns model behavior with capability boundaries through reward shaping that incentivizes refusal over futile reasoning and hindsight refusal augmentation that converts failures into refusal supervision. Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties, effectively achieving capability-aligned behavior without sacrificing utility. \footnote{https://github.com/icip-cas/Knowing-When-to-Quit}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。