让视觉文档模型学会分步推理，提升长文档理解能力。
Internalized Reasoning for Long-Context Visual Document Understanding
- 用合成数据生成分步思考轨迹，按页面相关性排序证据。
- 在Qwen3 VL上达58.3分，优于更大模型的57.0分。
- 推理过程内化后输出量减少12.4倍，适合资源受限场景。
视觉长文档理解对企业和法律、科研应用至关重要，但现有最佳方法未充分探索推理能力，而推理已推动数学与代码任务性能飞跃。本文提出一种合成数据流水线，通过评分每页与问题的相关性，提取文本证据并按相关性降序排列，生成思考轨迹。将这些轨迹在<think>标签内以SFT微调，并由<cot>控制令牌触发，再通过低强度模型合并实现推理能力内化。在Qwen3 VL 32B和Mistral Small 3.1 24B上进行实验：使用Qwen3 VL时，在MMLongBenchDoc上达到58.3分，超越7倍更大的Qwen3 VL 235B A22B（57.0分）；使用Mistral时，合成推理比从思维版本轨迹蒸馏出的结果高3.8分（在MMLBD-C上），且内化推理平均输出token数仅为显式推理的1/12.4。代码与数据已公开，支持复现与进一步研究。
原文摘要 · Abstract (English)
Visual long-document understanding is critical for enterprise, legal, and scientific applications, yet the best performing open recipes have not explored reasoning, a capability which has driven leaps in math and code performance. We introduce a synthetic data pipeline for reasoning in long-document understanding that generates thinking traces by scoring each page for question relevance, extracting textual evidence and ordering it from most to least relevant. We apply SFT to the resulting traces within \texttt{<think>} tags, gated by a \texttt{<cot>} control token, and the resulting reasoning capability is internalized via low-strength model merging. We study Qwen3 VL 32B and Mistral Small 3.1 24B. With Qwen3 VL, we achieve 58.3 on MMLongBenchDoc, surpassing the 7$\times$ larger Qwen3 VL 235B A22B (57.0). With Mistral, we show that synthetic reasoning outperforms distillation from the Thinking version's traces by 3.8 points on MMLBD-C, and internalized reasoning exhibits 12.4$\times$ fewer mean output tokens compared to explicit reasoning. We release our pipeline for reproducibility and further exploration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。