用程序推理提升图表摘要的准确性与效率
From Data to Insights: Exploring Program-of-Thoughts Prompting for Chart Summarization
- 通过编程中间件实现零样本推理，让轻量级视觉模型做数值计算
- 在多个指标上达到现有方法水平，且计算开销更低
- 适合需要高准确率与低资源消耗的图表自动化分析场景
图表通过结构化视觉形式传达数据洞察，但语义理解与数值推理要求使其摘要生成成为挑战。尽管视觉语言模型（VLMs）取得进展，现有方法缺乏统计事实验证机制且计算成本高。本文探索一种零样本策略，利用Python程序作为中间媒介，驱动轻量级VLM进行计算推理，以获取有效的图表统计信息。我们提出一种新的图表到字典的辅助任务，相比传统表格生成更灵活，特别适合与程序思维（PoT）策略结合。实验表明，该方法在语义和事实性指标上均达到现有方法水平。代码已公开于 https://anonymous.4open.science/r/ZeroShot-PoT-C2T-5A6B。
原文摘要 · Abstract (English)
Charts play a critical role in conveying numerical data insights through structured visual representations. However, semantic visual understanding and numerical reasoning requirements hinder the accurate description of charts, interpreting a challenging task in chart summarization. Despite recent advancements in visual language models (VLMs), approaches lack robust mechanisms for verifying statistical fact correctness and are computationally heavy. To address this gap, this paper explores a strategy of using zero-shot learning to motivate the lightweight VLMs to perform computational reasoning, via Python programs as intermediaries to derive valid summary statistics for chart understanding. Specifically, we introduce a novel chart-to-dictionary auxiliary task, offering a more flexible representation compared to traditional chart-to-table methods, making it particularly well-suited for integration with the Program-of-Thought (PoT) strategy. Experimental results demonstrate our strategy performs on par with existing chart summarization methods across semantic and factual metrics. Code is available on https://anonymous.4open.science/r/ZeroShot-PoT-C2T-5A6B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。