让多模态大模型学会专注关键信息，提升理解能力。
Reinforced Attention Learning
- 直接优化模型内部注意力分布，而非生成内容
- 在图像和视频任务上显著优于GRPO等基线方法
- 适合需要精准感知与跨模态对齐的研究者
通过强化学习进行后训练已显著提升大语言模型的推理能力，但将其应用于多模态大语言模型（MLLMs）时，依赖冗长推理过程的效果有限，甚至可能降低性能。本文提出受强化学习驱动的注意力学习（RAL），一种基于策略梯度的框架，直接优化模型内部的注意力分布，而非输出词元序列。通过将优化目标从‘生成什么’转向‘关注何处’，RAL促进复杂多模态输入中有效信息分配，增强模型的语义接地能力。在多个图像与视频基准测试中，RAL均稳定优于GRPO及其他基线方法。此外，我们提出就地注意力蒸馏（On-Policy Attention Distillation），证明迁移隐式注意力行为比传统知识蒸馏带来更强的跨模态对齐效果。结果表明，注意力策略是多模态后训练的一种原则性且通用的替代方案。
原文摘要 · Abstract (English)
Post-training with Reinforcement Learning (RL) has substantially improved reasoning in Large Language Models (LLMs) via test-time scaling. However, extending this paradigm to Multimodal LLMs (MLLMs) through verbose rationales yields limited gains for perception and can even degrade performance. We propose Reinforced Attention Learning (RAL), a policy-gradient framework that directly optimizes internal attention distributions rather than output token sequences. By shifting optimization from what to generate to where to attend, RAL promotes effective information allocation and improved grounding in complex multimodal inputs. Experiments across diverse image and video benchmarks show consistent gains over GRPO and other baselines. We further introduce On-Policy Attention Distillation, demonstrating that transferring latent attention behaviors yields stronger cross-modal alignment than standard knowledge distillation. Our results position attention policies as a principled and general alternative for multimodal post-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。