让AI根据用户意图,从长文档中自动生成精准图表。
Doc2Chart: Intent-Driven Zero-Shot Chart Generation from Documents
- 分两阶段处理:先解析意图提取数据,再选图类型生成代码
- 在金融与科学领域数据集上,图表准确率提升9分,类型匹配提升17分
- 无需标注数据,适合实际文档分析场景,尤其适合非技术用户
大型语言模型(LLMs)在通过指令微调将文本描述或表格转为可视化图表方面表现优异。然而,在真实场景中,用户往往需要从长文档中基于特定意图自动生成图表,而非手动筛选内容。为此,我们提出“基于意图的文档图表生成”任务:给定用户意图和文档,目标是在零样本设置下生成符合意图且基于文档内容的图表。我们提出一种无监督、两阶段框架:首先,通过分解意图并迭代验证与修正,由大模型从文档中提取相关信息;其次,采用启发式引导模块选择合适的图表类型,并生成最终代码。为评估生成图表的数据准确性,我们设计了一种基于归因的度量方法,利用结构化文本表示图表,而非依赖常失效的视觉解码指标。为验证方法有效性,我们构建了一个包含1,242个<意图, 文档, 图表>三元组的数据集,涵盖金融与科学两个领域,不同于现有数据集多集中于平行文本/表格与图表。与单次生成和基于查询的检索基线相比,我们的方法在图表数据准确率上最高提升9分,在图表类型匹配上提升17分。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated strong capabilities in transforming text descriptions or tables to data visualizations via instruction-tuning methods. However, it is not straightforward to apply these methods directly for a more real-world use case of visualizing data from long documents based on user-given intents, as opposed to the user pre-selecting the relevant content manually. We introduce the task of intent-based chart generation from documents: given a user-specified intent and document(s), the goal is to generate a chart adhering to the intent and grounded on the document(s) in a zero-shot setting. We propose an unsupervised, two-staged framework in which an LLM first extracts relevant information from the document(s) by decomposing the intent and iteratively validates and refines this data. Next, a heuristic-guided module selects an appropriate chart type before final code generation. To assess the data accuracy of the generated charts, we propose an attribution-based metric that uses a structured textual representation of charts, instead of relying on visual decoding metrics that often fail to capture the chart data effectively. To validate our approach, we curate a dataset comprising of 1,242 $<$intent, document, charts$>$ tuples from two domains, finance and scientific, in contrast to the existing datasets that are largely limited to parallel text descriptions/ tables and their corresponding charts. We compare our approach with baselines using single-shot chart generation using LLMs and query-based retrieval methods; our method outperforms by upto $9$ points and $17$ points in terms of chart data accuracy and chart type respectively over the best baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。