arXiv:2503.03669cs.CLcs.AI2025-03被引 1

用结构化推理问答提升大模型指令遵循能力,解决多轮对话中的执行偏差问题。

Attentive Reasoning Queries: A Systematic Method for Optimizing Instruction-Following in Large Language Models

  • 设计领域专用推理模板,通过针对性提问引导模型逐步遵循指令。
  • 在87个场景中达成90.2%成功率,优于思维链(86.1%)和直接生成(81.5%)。
  • 特别有效防止规则遗漏和幻觉,适合高可靠性业务场景使用。

我们提出了一种名为注意力推理查询(Attentive Reasoning Queries, ARQs)的新型结构化推理方法,显著提升大语言模型在复杂指令下的遵循能力。尽管大模型在多样化任务中表现出色,但在多轮对话中常无法持续遵守特定使用场景的复杂指令,影响关键业务应用。ARQs通过引入针对性的中间查询,引导模型在生成过程中反复确认核心指令并完成阶段性推理,从而增强指令保持性。在专为可靠客户交互代理设计的Parlant系统中,经87个测试场景验证,ARQs达到90.2%的成功率,优于思维链(86.1%)与直接生成(81.5%)。分析表明,该方法在处理指南重应用与幻觉抑制等顽固失败模式上表现突出,且合理设计下可能比自由形式推理更高效。结果证明,结构化推理是控制大模型在复杂场景中信息处理与决策的有效机制。

原文摘要 · Abstract (English)

We present Attentive Reasoning Queries (ARQs), a novel structured reasoning approach that significantly improves instruction-following in Large Language Models through domain-specialized reasoning blueprints. While LLMs demonstrate remarkable capabilities across diverse tasks, they often fail to maintain adherence to complex, use-case-specific instructions during multi-turn conversations, presenting challenges for business-critical applications. ARQs address this limitation by guiding LLMs through systematic reasoning steps with targeted queries that reinstate critical instructions and facilitate intermediate reasoning throughout the completion process. In extensive testing within Parlant, our framework for reliable customer-facing agents in which ARQs were born out of necessity, they achieved a 90.2% success rate across 87 test scenarios, outperforming both Chain-of-Thought reasoning (86.1%) and direct response generation (81.5%). ARQs showed particular strength in addressing persistent failure modes like guideline re-application and hallucination prevention. Our analysis also revealed that ARQs can potentially be more computationally efficient than free-form reasoning when carefully designed. These findings demonstrate that structured reasoning approaches provide effective mechanisms for controlling how LLMs process information and make decisions in complex scenarios.

指令遵循结构化推理大模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。