通过角色对抗式提示,让模型自动发现并修正自身错误。
FOR-Prompting: From Objection to Revision via an Asymmetric Prompting Protocol
- 设计防御者、质疑者、主持人三角色,由质疑者提出问题但不给答案。
- 在GSM8K上准确率媲美CoT,小模型上比直接提示提升显著。
- 无需训练或修改模型,适合本地部署和资源受限场景。
现有推理协议如思维链(CoT)和思维树(ToT)虽能组织内部思考,却缺乏外部质疑以触发自我修正的机制。本文提出异构提示协议FOR-Prompting(从质疑到修正),包含防御者、质疑者(提问者)与可选主持人角色:防御者给出答案,质疑者以问题形式提出异议但不提供解决方案,主持人可选地整合最终输出。在GSM8K数据集上,FOR-Prompting在相同模型基础上达到与CoT相当的准确率,并持续优于单次提示;在小型开源模型(如LLaMA-3.2-1B)上,显著优于直接提示,表现接近轻量级推理基线,展现其在低资源与本地设备部署中的潜力。跨模型角色互换实验表明,性能主要取决于防御者角色,使小模型也能有效担任提问者。该方法不仅适用于结构化数学任务,在开放式与多阶段任务中也促进更深入探索、更广覆盖与更高特异性。盲测显示,人类用户更偏好其在行程规划中的输出,优于强基线模型。该协议完全基于角色化提示,无需训练、不依赖模型内部访问或对称强代理,具备模型无关性,为可扩展的质疑驱动推理研究提供了实用路径,适用于云端与本地大模型的自动化迭代优化。
原文摘要 · Abstract (English)
Reasoning protocols such as Chain of Thought (CoT) and Tree of Thought (ToT) organize internal deliberation but lack an explicit mechanism for external questioning that elicits self-revision. We present FOR-Prompting (From Objection to Revision Prompting), an asymmetric protocol where a Defender proposes an answer, an Debater (Questioner) raises question-style objections with no direct fixes, and a Host optionally synthesizes the final output. Across GSM8K, FOR-Prompting matches the accuracy of CoT and consistently improves over single-prompting when evaluated under identical model backbones. On small-scale open-source models (e.g., LLaMA-3.2-1B), FOR-Prompting yields substantial gains over direct prompting and performs comparably to lightweight reasoning baselines, highlighting its promise for low-resource and on-device settings. Cross-model role-swapping further shows that performance is primarily determined by the Defender, enabling small models to act effectively as Questioners. Beyond structured math tasks, FOR-Prompting supports refinement in open-ended and multi-stage tasks: qualitative analysis shows improved exploration, coverage, and specificity, and a blind study of human preferences found that participants preferred FOR-Prompting outputs over strong LLM baselines in an itinerary-planning scenario. The protocol is model-agnostic and operates purely through role-structured prompting, requiring no training, access to model internals, or symmetrically strong agents. FOR-Prompting therefore enables scalable study of objection-driven reasoning and offers a practical mechanism for automated iterative refinement across both hosted and local LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。