用视觉语言模型提升表格结构识别，兼顾图文语义。
UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition
- 分而治之：先分割单元格，再用物理与逻辑解码器重建结构。
- 在PubTabNet等4个数据集上达新最佳，准确率显著提升。
- 引入视觉与语言引导器，增强对图文内容的理解能力。
在数字化时代，表格结构识别技术是处理和分析大量表格数据的关键工具。以往方法主要关注表格的视觉结构恢复，但往往无法有效理解表格中描述性文本单元格的语义。本文提出UniTabNet，一种基于图像到文本模型的新型表格结构解析框架。该框架采用‘分而治之’策略，利用图像到文本模型分离表格单元格，并结合物理解码器与逻辑解码器重构完整表格结构。我们进一步引入视觉引导器，引导模型聚焦关键区域，提升预测准确性；同时设计语言引导器，增强模型对表格图像中文本语义的理解能力。在PubTabNet、PubTables1M、WTW和iFLYTAB等主流表格结构数据集上的评估表明，UniTabNet达到新的性能上限，验证了该方法的有效性。代码将公开发布。
原文摘要 · Abstract (English)
In the digital era, table structure recognition technology is a critical tool for processing and analyzing large volumes of tabular data. Previous methods primarily focus on visual aspects of table structure recovery but often fail to effectively comprehend the textual semantics within tables, particularly for descriptive textual cells. In this paper, we introduce UniTabNet, a novel framework for table structure parsing based on the image-to-text model. UniTabNet employs a ``divide-and-conquer'' strategy, utilizing an image-to-text model to decouple table cells and integrating both physical and logical decoders to reconstruct the complete table structure. We further enhance our framework with the Vision Guider, which directs the model's focus towards pertinent areas, thereby boosting prediction accuracy. Additionally, we introduce the Language Guider to refine the model's capability to understand textual semantics in table images. Evaluated on prominent table structure datasets such as PubTabNet, PubTables1M, WTW, and iFLYTAB, UniTabNet achieves a new state-of-the-art performance, demonstrating the efficacy of our approach. The code will also be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。