让视频分类模型从模仿走向直觉,提升复杂场景下的泛化能力
From Imitation to Intuition: Intrinsic Reasoning for Open-Instance Video Classification
- 通过冷启动对齐与强化学习优化视觉语言模型的内在推理能力
- 在开放实例视频分类任务中准确率显著超越传统方法
- 适合需要强泛化能力的现实视频理解场景
传统视频分类模型在同质数据分布下表现良好,但在真实世界中面临开放实例挑战——类内差异大且复杂,超出现有基准。尽管视觉语言模型(VLMs)具备更强泛化性,却未充分挖掘其推理能力。本文提出DeepIntuit框架,将开放实例视频分类从模仿推进至内在推理。该方法首先通过监督对齐初始化推理能力,再利用组相对策略优化(GRPO)进行强化学习,提升推理一致性;最后引入直觉校准阶段,以训练分类器捕捉经优化的VLM推理轨迹,实现稳定知识迁移且无分布偏移。大量实验表明,深度依赖内在推理的DeepIntuit在开放实例视频分类中显著优于单纯特征模仿的方法。
原文摘要 · Abstract (English)
Conventional video classification models, acting as effective imitators, excel in scenarios with homogeneous data distributions. However, real-world applications often present an open-instance challenge, where intra-class variations are vast and complex, beyond existing benchmarks. While traditional video encoder models struggle to fit these diverse distributions, vision-language models (VLMs) offer superior generalization but have not fully leveraged their reasoning capabilities (intuition) for such tasks. In this paper, we bridge this gap with an intrinsic reasoning framework that evolves open-instance video classification from imitation to intuition. Our approach, namely DeepIntuit, begins with a cold-start supervised alignment to initialize reasoning capability, followed by refinement using Group Relative Policy Optimization (GRPO) to enhance reasoning coherence through reinforcement learning. Crucially, to translate this reasoning into accurate classification, DeepIntuit then introduces an intuitive calibration stage. In this stage, a classifier is trained on this intrinsic reasoning traces generated by the refined VLM, ensuring stable knowledge transfer without distribution mismatch. Extensive experiments demonstrate that for open-instance video classification, DeepIntuit benefits significantly from transcending simple feature imitation and evolving toward intrinsic reasoning. Our project is available at https://bwgzk-keke.github.io/DeepIntuit/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。