让编程助手记住用户习惯,减少重复提问。
Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants

- 基于用户历史会话识别个性化模糊模式,自动修复常见歧义。
- 在600个会话中验证,使用历史可降低70%澄清次数。
- 适合开发能长期学习用户偏好的智能编程助手。
AI辅助编程正将用户的非正式意图转化为可执行代码,但请求中常含重复出现的个性化歧义。现有方法多孤立处理当前会话中的歧义,依赖额外澄清。本文提出个性化歧义适应新任务:利用用户过往已解决会话,识别其重复出现的歧义模式,生成正确代码并最小化澄清。为此构建了CAPA基准,包含600个会话(60个用户-歧义组合),通过三阶段生成流程注入六类歧义机制。评估12个主流LLM在无历史与同用户历史条件下,以可执行成功率、首回合成功和完成轮次为指标。分析显示用户身份与历史记忆显著影响性能,提出轻量级同用户历史门控机制。结果表明,利用历史可有效提升代码对齐度,减少重复澄清。
原文摘要 · Abstract (English)
AI-assisted coding increasingly translates informal user intent into executable software, yet coding requests often contain ambiguities that recur in user-specific ways across tasks and sessions. Existing disambiguation methods typically address each ambiguous request in isolation within the current coding session, often through eliciting additional clarification. However, whether resolved session history from the same user can serve as memory for resolving recurring personalized ambiguity in a newly opened session remains underexplored. We formulate personalized ambiguity adaptation as a new task: given a user's previously resolved coding sessions and a new ambiguous request, an assistant should identify the recurring ambiguity pattern, produce the intended executable solution, and minimize clarification. To benchmark this task, we introduce CAPA, which characterizes personalized coding ambiguity through six mechanisms and injects these mechanisms into unambiguous executable tasks using a controlled three-stage generation pipeline. CAPA contains 600 coding sessions across 60 balanced user--ambiguity cells, including 300 held-out evaluation sessions. We evaluate 12 recent LLMs under no-history and same-user-history conditions using executable success, first-turn success, and turns-to-completion. Our analyses examine task difficulty, user identity, and memory-based history use, and we further propose same-user history gating as a lightweight inference-time method. CAPA provides a foundation for developing long-term coding assistants that better align generated code with user intent while reducing repeated clarification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。