不靠推理,直接对齐视觉与答案,让多模态文档问答更高效。
Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment

- 用直接对齐视觉特征和答案的方法替代中间推理步骤。
- 推理模型在训练中自我抑制推理痕迹,推理耗时降低60%以上。
- 适合追求高效推理、少数据训练的多模态问答研究者。
高效的多模态文档问答需精准定位支持答案的文档区域,但现有方法分两路:监督微调(SFT)依赖大量标注数据且易达性能瓶颈;以推理为核心的强化学习(RL)则依赖冗长中间轨迹,增加推理开销却无明显收益。本文提出感知型相对策略优化(Perception-RFT),采用组相对策略优化(GRPO)框架,在多模态文档问答中跳过中间推理步骤,直接将视觉特征对齐到结构化定位输出。为验证推理必要性,我们在相同奖励设置下构建了推理增强版本。结果发现,推理模型在40亿参数规模下训练时会主动抑制推理轨迹,收敛至仅依赖感知的策略,单次查询推理令牌数减少超60%,而含推理的强化学习表现反而劣于纯感知训练。通过对Qwen3-VL-4B的细粒度分析,确认文本领域中的微调饱和与冷启动强化学习不稳定现象同样存在于多模态场景,并揭示一种未被认识的“定位分歧”:在两个分布外(OOD)基准上(共4,828样本),联合优化下语义鲁棒性与几何精度存在选择性权衡。进一步表明,早期从SFT转到RL可实现相当精度,同时减少65%训练数据。
原文摘要 · Abstract (English)
Efficient multimodal document question answering with explicit visual grounding, locating the precise document region that supports each answer remains an open challenge. Current approaches bifurcate into Supervised Fine-Tuning (SFT), which requires large annotated datasets and reaches optimization plateaus, and reasoning-centric Reinforcement Learning (RL), which depends on verbose intermediate traces that inflate inference token cost without clear benefit. We introduce Perception-RFT, a training framework that applies Group Relative Policy Optimization (GRPO) to multimodal document QA, bypassing intermediate reasoning tokens to directly align visual features with structured grounding outputs. To rigorously evaluate the necessity of reasoning, we construct a reasoning variant under identical reward settings. We find that reasoning-enabled models suppress their reasoning traces during training, converging to direct perception-based policies at the 4B parameter scale, reducing per-query inference token length by more than 60%, while reasoning-enabled RL underperforms perception-only training. Through a fine-grained analysis of Qwen3-VL-4B optimization dynamics, we confirm that SFT saturation and cold-start RL instability established in text-domain post-training extend to multimodal, and identify a previously uncharacterized Grounding Divergence: a selective trade-off between semantic robustness and geometric precision on two out of distribution (OOD) benchmarks (4,828 samples) under joint RL optimization. We further show that an early SFT$\rightarrow$RL transition achieves comparable precision with 65% less training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。