arXiv:2604.00696cs.CV2026-04

无需标注数据,模型可在测试时自动适应新视频。

TTA-Vid: Generalized Test-Time Adaptation for Video Reasoning

  • 测试时通过多帧子集逐步推理并用频率奖励更新模型。
  • 单个样本训练后可泛化到整个数据集甚至跨数据集。
  • 适合无标注数据场景,尤其适合快速部署新视频任务。

近期视频理解模型在时空与多模态理解上表现优异,但依赖大规模有监督数据和多阶段训练流程,导致训练成本高且难适应新领域。本文提出一种基于测试时强化学习的视频理解自适应方法(TTA-Vid),使预训练模型能在测试阶段无需标签即可适应新视频输入。该方法结合两个组件:一是在推理时对多个帧子集进行逐步推理;二是基于跨帧子集的批量感知频率奖励作为伪真值来更新模型。实验表明,仅用单个批次或单个样本训练的模型,也能在测试时泛化至整个数据集乃至跨数据集。由于适配完全在测试阶段完成,无需真实标注或专用训练集。此外,还设计了基于多臂老虎机的自适应帧选择策略,根据相同奖励机制优先选取信息量高的帧。评估显示,TTA-Vid在多种视频理解任务中持续提升性能,并超越使用大规模数据训练的现有最优方法,凸显测试时强化学习在时序多模态理解中的潜力。

原文摘要 · Abstract (English)

Recent video reasoning models have shown strong results on temporal and multimodal understanding, yet they depend on large-scale supervised data and multi-stage training pipelines, making them costly to train and difficult to adapt to new domains. In this work, we leverage the paradigm of Test-Time Reinforcement Learning on video-language data to allow for adapting a pretrained model to incoming video samples at test-time without explicit labels. The proposed test-time adaptation for video approach (TTA-Vid) combines two components that work simultaneously: (1) a test-time adaptation that performs step-by-step reasoning at inference time on multiple frame subsets. We then use a batch-aware frequency-based reward computed across different frame subsets as pseudo ground truth to update the model. It shows that the resulting model trained on a single batch or even a single sample from a dataset, is able to generalize at test-time to the whole dataset and even across datasets. Because the adaptation occurs entirely at test time, our method requires no ground-truth annotations or dedicated training splits. Additionally, we propose a multi-armed bandit strategy for adaptive frame selection that learns to prioritize informative frames, guided by the same reward formulation. Our evaluation shows that TTA-Vid yields consistent improvements across various video reasoning tasks and is able to outperform current state-of-the-art methods trained on large-scale data. This highlights the potential of test-time reinforcement learning for temporal multimodal understanding.

视频理解测试时自适应强化学习无监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。