无需标注数据，让视觉语言模型自动学会识别表格。
TRivia: Self-supervised Fine-tuning of Vision-Language Models for Table Recognition
- 用自监督方式训练模型，从无标注表格图中学习
- 在三个基准上超越Gemini等商用模型
- 适合隐私敏感场景的开源表格识别方案
表格识别（TR）旨在将表格图像转化为半结构化表示（如HTML或Markdown）。作为文档解析的核心组件，传统方法依赖大量标注数据进行监督学习。尽管视觉语言模型（VLMs）显著提升了性能，但进一步突破仍需大规模标注数据，成本高昂。因此，虽有专有模型持续提升上限，但受限于资源与隐私法规，开源模型仍明显落后。为此，我们提出TRivia，一种基于自监督微调的方法，使预训练的VLM可直接从真实世界中的无标注表格图像中学习。该方法基于分组相对策略优化，自动识别最有利于学习的样本，并通过问答式奖励机制消除人工标注需求。注意力引导模块为每张表格生成多样化问题，模型能否正确解释识别结果并回答问题，成为优化反馈。这一闭环流程使模型能自主学习表格识别、结构化与推理能力，无需标注数据。基于此，我们推出了TRivia-3B——一个开源、轻量且性能领先的表格识别模型，在三个主流基准上优于现有系统（如Gemini 2.5 Pro、MinerU2.5）。模型与代码已发布于：https://github.com/HKU-TASR/TRivia
原文摘要 · Abstract (English)
Table recognition (TR) aims to transform table images into semi-structured representations such as HTML or Markdown. As a core component of document parsing, TR has long relied on supervised learning, with recent efforts dominated by fine-tuning vision-language models (VLMs) using labeled data. While VLMs have brought TR to the next level, pushing performance further demands large-scale labeled data that is costly to obtain. Consequently, although proprietary models have continuously pushed the performance boundary, open-source models, often trained with limited resources and, in practice, the only viable option for many due to privacy regulations, still lag far behind. To bridge this gap, we introduce TRivia, a self-supervised fine-tuning method that enables pretrained VLMs to learn TR directly from unlabeled table images in the wild. Built upon Group Relative Policy Optimization, TRivia automatically identifies unlabeled samples that most effectively facilitate learning and eliminates the need for human annotations through a question-answering-based reward mechanism. An attention-guided module generates diverse questions for each table image, and the ability to interpret the recognition results and answer them correctly provides feedback to optimize the TR model. This closed-loop process allows the TR model to autonomously learn to recognize, structure, and reason over tables without labeled data. Leveraging this pipeline, we present TRivia-3B, an open-sourced, compact, and state-of-the-art TR model that surpasses existing systems (e.g., Gemini 2.5 Pro, MinerU2.5) on three popular benchmarks. Model and code are released at: https://github.com/HKU-TASR/TRivia
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。