让AI先理解数据结构,再分析才能更准确。
Walk Before You Run: The Importance of Data Exploration for Data Analysis Agents

- 把数据探索设为独立评估环节,检验AI对表格逻辑的理解。
- 实测发现强模型仍会遗漏关键数据关系,影响后续分析正确率。
- 适合需要高可靠性的数据分析场景,尤其适合人机协作流程。
基于大语言模型的数据分析工具广泛用于处理混乱的电子表格和工作簿,从回答上传文件的问题到生成代码、摘要与可视化。当前系统多以最终任务答案的正确性作为评估标准,但可靠分析依赖于前期对数据集内容的理解。对于复杂工作簿,数据探索包括识别物理表背后的逻辑表、解释列语义、恢复键值与关系、检测质量问题。现有工具与评测中该步骤常被隐含,造成下游任务表现与实际数据理解能力之间的差距。本文首次揭示此被忽视的鸿沟,将数据探索列为首要评估目标,并通过实验表明强化探索支持可提升下游任务性能。为此,我们引入两个基准设置:基于维他命D研究数据的真实多表工作簿基准,以及扩展后的DSBench(增加固定模式的数据探索成果物)。在两者中,系统通过生成包含表、列、语义角色、关系及统计信号的结构化成果物进行评估。结果表明,即使强模型读取了原始内容,仍会遗漏重要逻辑结构;而显式的数据探索支持显著提升下游正确率,证明其应作为可检查的一阶流程,成为领域专家介入审核的自然节点。
原文摘要 · Abstract (English)
LLM-based data-analysis tools are increasingly used to help users analyze messy spreadsheets and workbooks, from answering questions over uploaded files to generating code, summaries, and visualizations. These systems are often evaluated by the correctness of their final downstream answers. However, reliable data analysis also depends on an earlier step: understanding what the dataset contains before solving the requested task. For complex workbooks, this Data Exploration step includes identifying the logical tables behind physical sheets, interpreting column semantics, recovering keys and relationships, and detecting quality issues. In current tools and benchmarks, this step is usually left implicit, creating a gap between downstream task performance and the dataset understanding needed for reliable, human-checkable analysis. Our key contribution is to identify this overlooked gap, make Data Exploration a first-class evaluation target, and show through downstream experiments that stronger Data Exploration support improves task performance. To evaluate dataset understanding directly, we introduce two benchmark settings: a real multi-sheet workbook benchmark based on a Vitamin D study dataset, and an extension of DSBench with schema-fixed Data Exploration artifacts. In both settings, systems are evaluated by the quality of a structured artifact capturing tables, columns, semantic roles, relationships, and profiling signals. Our results show that strong LLMs and data-analysis agents still miss important logical structure even when they read spreadsheet content. Furthermore, explicit Data Exploration support often improves downstream correctness, suggesting it should be treated as a first-class, inspectable stage in LLM data-analysis workflows and a natural human-in-the-loop checkpoint where domain experts can review and correct the artifact before downstream analysis proceeds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。