让表格问答可解释:从图像到代码的透明推理流程
ExpliCIT-QA: Explainable Code-Based Image Table Question Answering
- 用链式思维提取表格图像内容,生成分步自然语言推理
- 自动生成可执行的Python代码并反馈纠错,最终输出答案
- 全链路透明可审计,适合金融医疗等高可靠性场景
我们提出ExpliCIT-QA,一个将先前MRT方法扩展为多模态流水线的系统,能处理复杂表格图像并提供可解释的答案。该系统采用模块化设计:(1) 多模态表格理解,使用链式思维从表格图像中提取并转换内容;(2) 基于语言的推理,生成逐步自然语言解释以解决问题;(3) 自动代码生成,根据推理步骤生成Python/Pandas脚本,并通过反馈机制处理错误;(4) 代码执行计算最终答案;(5) 生成自然语言解释,说明答案的计算过程。系统设计注重透明性与可审计性:所有中间输出、解析表格、推理步骤、生成代码及最终答案均可被检查。该策略有效缩小了端到端表格视觉问答系统中的可解释性差距。我们在TableVQA-Bench基准上评估ExpliCIT-QA,对比现有基线,证明其在可解释性和透明性方面有显著提升,为金融、医疗等需审计的敏感领域应用打开可能。
原文摘要 · Abstract (English)
We present ExpliCIT-QA, a system that extends our previous MRT approach for tabular question answering into a multimodal pipeline capable of handling complex table images and providing explainable answers. ExpliCIT-QA follows a modular design, consisting of: (1) Multimodal Table Understanding, which uses a Chain-of-Thought approach to extract and transform content from table images; (2) Language-based Reasoning, where a step-by-step explanation in natural language is generated to solve the problem; (3) Automatic Code Generation, where Python/Pandas scripts are created based on the reasoning steps, with feedback for handling errors; (4) Code Execution to compute the final answer; and (5) Natural Language Explanation that describes how the answer was computed. The system is built for transparency and auditability: all intermediate outputs, parsed tables, reasoning steps, generated code, and final answers are available for inspection. This strategy works towards closing the explainability gap in end-to-end TableVQA systems. We evaluated ExpliCIT-QA on the TableVQA-Bench benchmark, comparing it with existing baselines. We demonstrated improvements in interpretability and transparency, which open the door for applications in sensitive domains like finance and healthcare where auditing results are critical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。