arXiv:2504.16331cs.SEcs.LG2025-04被引 7

让代码大模型学会主动提问澄清需求,提升开发可靠性。

Can Code Language Models Learn Clarification-Seeking Behaviors?

  • 通过合成数据与指令微调,训练模型识别模糊需求
  • 在模糊任务中提问率提升至63%,优质问题占比52%
  • 适合需要高可靠性的代码生成场景,如协作AI开发

大型语言模型在代码生成任务中表现出色,但其输出仍与人类开发者的问题解决策略存在差距。与人类通过迭代对话澄清需求不同,当前模型常在需求不明确时直接生成代码,导致结果不可靠。本文提出ClarifyCoder框架,通过合成数据生成和指令微调,使代码大模型具备识别模糊性并请求澄清的能力。该框架包含两项核心技术:(1)在编程数据集中引入需澄清的场景,生成具备澄清意识的训练数据;(2)设计微调策略,使模型在面对不完整或模糊需求时优先寻求澄清而非立即生成代码。我们还对整合标准微调进行联合优化进行了实证分析。实验表明,在模糊任务上,ClarifyCoder实现了63%的沟通率(绝对提升40%)和52%的良好问题率(绝对提升30%),显著提升了模型的沟通能力,同时保持了良好的代码生成性能。

原文摘要 · Abstract (English)

Large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks. However, a gap remains between their output and the problem-solving strategies of human developers. Unlike humans, who spend substantial time disambiguating requirements through iterative dialogue, LLMs often generate code despite ambiguities in natural language requirements, leading to unreliable solutions. Different from prior work, we study whether a Code LLM can be fine-tuned to learn clarification-seeking behavior. While recent work has focused on LLM-based agents for iterative code generation, we argue that the ability to recognize and query ambiguous requirements should be intrinsic to the models themselves, especially in agentic AI where models and humans collaborate. We present ClarifyCoder, a framework with synthetic data generation and instruction-tuning that fine-tunes an LLM to identify ambiguities and request clarification before code generation. Our approach has two components: (1) a data synthesis technique that augments programming datasets with scenarios requiring clarification to generate clarification-aware training data, and (2) a fine-tuning strategy that teaches models to prioritize seeking clarification over immediate code generation when faced with incomplete or ambiguous requirements. We also provide an empirical analysis of integrating ClarifyCoder with standard fine-tuning for joint optimization of clarification-awareness and coding ability. Experimental results show that ClarifyCoder achieves a 63% communication rate (40% absolute increase) and a 52% good question rate (30% absolute increase) on ambiguous tasks, significantly improving LLMs' communication capabilities while maintaining code generation performance.

代码生成澄清行为大模型指令微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。