arXiv:2509.14671cs.CLcs.AI2025-09中稿 · ICLR被引 7

动态选择文本或图像路径,提升表格理解准确率与效率。

TableDART: Dynamic Adaptive Multi-Modal Routing for Table Understanding

  • 用轻量门控网络动态选最优模态路径,避免冗余冲突。
  • 在7个基准上平均超越最强基线4.02%,达开源模型新高。
  • 无需微调大模型,适合资源有限的表格理解任务。

表格数据的语义与结构建模仍是有效表格理解的核心挑战。现有表转文本方法将表格展平输入大语言模型(LLMs),但丢失关键结构信息;表转图像方法保留结构却难以捕捉精确语义。近期的多模态策略尝试结合文本与视觉视图,但存在两个问题:(1) 在大型多模态大模型(MLLMs)中对每对查询-表格静态处理双模态,导致冗余甚至冲突;(2) 依赖昂贵的MLLM微调。为此,我们提出TableDART,一个训练高效的框架,通过复用预训练单模态模型融合多模态视图。TableDART引入一个仅259万参数的轻量级MLP门控网络,动态为每对表-查询选择最优路径(仅文本、仅图像或融合),减少冗余并避免因文本与视觉线索不一致引发的冲突。通过路由至最适视图,框架同时提升准确率与效率。此外,我们设计新型代理,通过分析文本与图像模型输出,选择最优结果或通过推理合成新答案,避免全模型微调的高昂成本。在7个基准上的大量实验表明,TableDART在开源模型中达到新的最先进性能,平均超越最强基线4.02%。代码已公开:https://github.com/xiaobo-xing/TableDART。

原文摘要 · Abstract (English)

Modeling semantic and structural information from tabular data remains a core challenge for effective table understanding. Existing Table-as-Text approaches flatten tables for large language models (LLMs), but lose crucial structural cues, while Table-as-Image methods preserve structure yet struggle with precise semantics. Recent Table-as-Multimodality strategies attempt to combine textual and visual views, but they (1) statically process both modalities for every query-table pair within large multimodal LLMs (MLLMs), inevitably introducing redundancy and even conflicts, and (2) depend on costly fine-tuning of MLLMs. In light of this, we propose TableDART, a training-efficient framework that integrates multimodal views by reusing pretrained single-modality models. TableDART introduces a lightweight 2.59M-parameter MLP gating network that dynamically selects the optimal path (Text-only, Image-only, or Fusion) for each table-query pair, reducing redundancy and avoiding conflicts that arise when textual and visual views of the same table provide inconsistent cues. By routing to the most appropriate view, our framework improves both accuracy and efficiency. In addition, we propose a novel agent to mediate cross-modal knowledge integration by analyzing outputs from text- and image-based models, either selecting the best result or synthesizing a new answer through reasoning. This design avoids the prohibitive costs of full MLLM fine-tuning. Extensive experiments on seven benchmarks show that TableDART establishes new state-of-the-art performance among open-source models, surpassing the strongest baseline by an average of 4.02%. The code is available at: https://github.com/xiaobo-xing/TableDART.

表格理解多模态动态路由

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。