通过对话澄清用户模糊的可视化需求，提升代码生成准确性
Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation
- 构建可视化任务中模糊性的分类体系与量化指标
- 实验证明新指标比基线更贴近人工标注结果
- 多轮对话可显著降低歧义，适合交互式代码生成场景
人机沟通中目标共识至关重要，但自然语言的模糊性常导致生成代码看似正确却偏离意图。本文聚焦数据可视化领域，分析自然语言描述中引发代码生成偏差的多种模糊类型。利用DS-1000数据集中的Matplotlib问题，提出一套可量化的模糊性指标，实证显示其与人工标注的相关性优于现有不确定性基线。进一步研究多轮对话如何缓解模糊性：基于格赖斯合作原则、话语表征理论和讨论中的问题三种策略，模拟用户研究发现，有意识的对话能有效减少歧义，提升代码准确率，凸显多轮交互在代码生成中的价值。
原文摘要 · Abstract (English)
Establishing shared goals is a fundamental step in human-AI communication. However, ambiguities can lead to outputs that seem correct but fail to reflect the speaker's intent. In this paper, we explore this issue with a focus on the data visualization domain, where ambiguities in natural language impact the generation of code that visualizes data. The availability of multiple views on the contextual (e.g., the intended plot and the code rendering the plot) allows for a unique and comprehensive analysis of diverse ambiguity types. We develop a taxonomy of types of ambiguity that arise in this task and propose metrics to quantify them. Using Matplotlib problems from the DS-1000 dataset, we demonstrate that our ambiguity metrics better correlate with human annotations than uncertainty baselines. Our work also explores how multi-turn dialogue can reduce ambiguity, therefore, improve code accuracy by better matching user goals. We evaluate three pragmatic models to inform our dialogue strategies: Gricean Cooperativity, Discourse Representation Theory, and Questions under Discussion. A simulated user study reveals how pragmatic dialogues reduce ambiguity and enhance code accuracy, highlighting the value of multi-turn exchanges in code generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。