arXiv:2602.23823cs.CV2026-02被引 1

用注意力引导强化学习,低成本提升视频理解中的细节感知能力

APPO: Attention-guided Perception Policy Optimization for Video Reasoning

  • 通过令牌级密集奖励优化关键帧的感知令牌,提升细粒度视觉理解
  • 在3/7B模型上,性能比GRPO和DAPO高出0.5%~4%
  • 无需昂贵标注,适合资源有限但需强感知的视频推理场景

复杂视频推理实际上过度依赖细粒度感知,而非专家级推理能力。实证观察发现,当感知能力接近固定时,将推理模型从Qwen3-8B升级到OpenAI-o3仅带来0.7%的性能提升;而感知模型规模从7B增至32B即可提升1.4%,说明增强感知比改进推理更关键。因此,探索如何通过推理过程提升感知能力,且无需昂贵的细粒度标注,具有重要意义。为此,我们提出注意力引导的感知策略优化算法APPO,利用令牌级密集奖励优化同一关键视频帧上的感知令牌(称为组内感知令牌)。在多种视频基准和不同规模模型(3/7B)上的实验表明,APPO始终优于GRPO和DAPO,性能提升0.5%~4%。本工作为以低成本方式通过推理增强模型感知能力提供了可行路径,适用于多样化的实际场景需求。

原文摘要 · Abstract (English)

Complex video reasoning, actually, relies excessively on fine-grained perception rather than on expert (e.g., Ph.D, Science)-level reasoning. Through extensive empirical observation, we have recognized the critical impact of perception. In particular, when perception ability is almost fixed, enhancing reasoning from Qwen3-8B to OpenAI-o3 yields only 0.7% performance improvement. Conversely, even minimal change in perception model scale (from 7B to 32B) boosts performance by 1.4%, indicating enhancing perception, rather than reasoning, is more critical to improve performance. Therefore, exploring how to enhance perception ability through reasoning without the need for expensive fine-grained annotation information is worthwhile. To achieve this goal, we specially propose APPO, the Attention-guided Perception Policy Optimization algorithm that leverages token-level dense rewards to improve model's fine-grained perception. The core idea behind APPO is to optimize those tokens from different responses that primarily focus on the same crucial video frame (called intra-group perception tokens). Experimental results on diverse video benchmarks and models with different scales (3/7B) demonstrate APPO consistently outperforms GRPO and DAPO (0.5%~4%). We hope our work provides a promising approach to effectively enhance model's perception abilities through reasoning in a low-cost manner, serving diverse scenarios and demands.

视频推理感知优化强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。