让编程大模型学会主动提问，解决指令不全问题。
Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents
- 用多智能体架构分离识别模糊和写代码，判断何时该提问。
- 在未完全指定任务上达成69.40%成功率，接近完整指令表现。
- 复杂任务主动问，简单任务少问，提问行为更合理。
随着大语言模型代理在软件工程等开放领域应用增多，它们常遇到缺少关键上下文的模糊指令。人类开发者会主动提问澄清，但现有代理主要优化自主执行。本文系统评估了大模型代理在未完全指定的SWE-bench Verified变体上的澄清能力。提出一种不确定性感知的多智能体框架，将模糊性检测与代码执行解耦。在多种专有及开源前沿LLM上，该框架实现69.40%的任务解决率，显著优于标准单智能体设置，并缩小了与完整指令下代理的性能差距。此外，多智能体系统表现出校准良好的信息获取行为：简单任务减少提问，复杂任务主动求解。结果表明，当前模型可转变为积极协作伙伴，在真实模糊任务中自主判断何时提问以获取缺失信息。
原文摘要 · Abstract (English)
As Large Language Model (LLM) agents are increasingly deployed in open-ended domains like software engineering, they frequently encounter underspecified instructions that lack crucial context. While human developers naturally resolve underspecification by asking clarifying questions, current agents are largely optimized for autonomous execution. In this work, we systematically evaluate the clarification-seeking abilities of LLM agents on an underspecified variant of SWE-bench Verified. We propose an uncertainty-aware multi-agent scaffold that decouples underspecification detection from code execution. Across both proprietary and open-weight frontier LLMs, our scaffold achieves a 69.40% task resolve rate, significantly outperforming a standard single-agent setup and closing the performance gap with agents operating on fully specified instructions. Furthermore, we find that the multi-agent system exhibits well-calibrated information-seeking behavior, conserving queries on simple tasks while proactively seeking information on more complex issues. These findings indicate that current models can be turned into proactive collaborators, where agents independently recognize when to ask questions to elicit missing information in real-world, underspecified tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。