分离表格结构与内容，用轻量方法提升视觉语言模型的表格推理能力
Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance
- 将表格结构与语义信息解耦对齐，避免信息混淆
- 在多个基准上显著提升模型对未见表格结构的泛化能力
- 无需外部工具或大量标注，适合资源受限场景
表格图像的推理对大型视觉语言模型（LVLMs）仍具挑战性，源于复杂布局及结构与内容信息的高度耦合。现有方法常依赖昂贵的监督训练、强化学习或外部工具，限制了效率与可扩展性。本文提出核心问题：如何在极少标注且无外部工具的前提下，使LVLM适应表格推理？为此，我们首先提出DiSCo框架，通过在多模态对齐中显式分离结构抽象与语义定位，实现高效适配。在此基础上，进一步构建Table-GLS框架，通过全局到局部的结构引导，实现结构化探索与证据驱动推理。跨多个基准的实验证明，该框架能有效增强LVLM的表格理解与推理能力，尤其在未见表格结构上表现出强泛化性。数据与代码已公开于https://github.com/AAAndy-Zhu/TableVLM。
原文摘要 · Abstract (English)
Reasoning over table images remains challenging for Large Vision-Language Models (LVLMs) due to complex layouts and tightly coupled structure-content information. Existing solutions often depend on expensive supervised training, reinforcement learning, or external tools, limiting efficiency and scalability. This work addresses a key question: how to adapt LVLMs to table reasoning with minimal annotation and no external tools? Specifically, we first introduce DiSCo, a Disentangled Structure-Content alignment framework that explicitly separates structural abstraction from semantic grounding during multimodal alignment, efficiently adapting LVLMs to tables structures. Building on DiSCo, we further present Table-GLS, a Global-to-Local Structure-guided reasoning framework that performs table reasoning via structured exploration and evidence-grounded inference. Extensive experiments across diverse benchmarks demonstrate that our framework efficiently enhances LVLM's table understanding and reasoning capabilities, particularly generalizing to unseen table structures. Our data and code are available at https://github.com/AAAndy-Zhu/TableVLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。