直接从文档图像端到端提取键值对，省去OCR步骤，更快更小。
Identify, Locate, Link: End-to-End Key-Value Extraction from Document Images
- 用小型视觉语言模型直接解析图像，一步完成识别、定位和关联。
- 在多个数据集上超越更大模型，小模型性能接近大模型。
- 适合需要轻量高效文档信息提取的工业应用。
传统文档处理流程将OCR与下游模型串联，导致多阶段误差传播。我们微调了一个256M参数的紧凑视觉语言模型SmolDocling，实现从文档图像中端到端的键值提取，无需OCR预处理，一次通过完成识别、定位与关联。扩展DocTags引入专门的键、值、区域和链接标签，支持多对多关系的统一输出序列。针对数据有限问题，设计了结合合成表单填充与基于图的裁剪的增强管道，保留完整的键值子图。进一步提出布局感知评估框架，将文本匹配与空间边界框验证结合。在FUNSD、XFUND及大规模私有数据集上，该模型在布局感知评估下优于更大的零样本VLM基线，体积仅为Qwen2.5-VL（7B）的1/27，推理速度超过5倍快。模型权重将在发表后公开。
原文摘要 · Abstract (English)
Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information extraction, leading to multi-stage error propagation. We fine-tune SmolDocling, a compact 256M-parameter vision-language model (VLM), to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing. We extend DocTags with specialized key, value, region, and link tags, enabling many-to-many relationships in a unified output sequence. To address data limitations, we design an augmentation pipeline combining synthetic form filling and graph-based crops that preserve complete key-value subgraphs. We further introduce a layout-aware evaluation framework extending text matching with spatial bounding box verification. On FUNSD, XFUND, and a large-scale private dataset, our model outperforms larger zero-shot VLM baselines under layout-aware evaluation, while being 27 times smaller than Qwen2.5-VL (7B) and over 5 times faster at inference. The model weights will be released publicly after publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。