用冻结验证器指导视频推理模型自我优化，提升证据选择能力。
Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning
- 多智能体框架让定位器采样候选片段，验证器打分并反馈
- 零样本跨任务迁移，视频问答准确率达54.1%，时间定位达46.1%IoU
- 适合需要精准时间证据选择的视频理解场景
视频推理任务如带语境的视频问答和时间定位，需选出支持问题的时间证据。当前多数训练采用边界回归或片段生成等局部目标，验证仅用于推理阶段重排序。本文研究冻结验证器能否作为训练信号。提出多智能体框架：可训练的定位器采样候选轨迹与证据片段，冻结的验证器赋予查询相关的片段得分，基于组内相对策略梯度的目标偏好优于同输入内同伴的轨迹，自举校准损失引导时间预测向验证器偏好的片段靠拢。在源任务上训练，无需目标数据集微调，两亿参数实例在零样本下跨带语境问答、时间定位、长视频问答任务表现良好，在带语境问答基准上达到28.7% IoU和25.4%答案定位准确率，时间定位基准达46.1% IoU，长视频问答达54.1%。相比强基线，提升虽小但稳定，尤其在交并比和中等重叠召回等相关性指标上更明显。结果支持冻结验证作为证据选择的训练信号，但严格边界精度仍较弱。代码与模型见https://anonymous.4open.science/r/MASIRL-E50C/
原文摘要 · Abstract (English)
Video reasoning tasks such as grounded video question answering and temporal grounding require selecting temporal evidence that supports the query. In many current training setups, temporal supervision is applied through local objectives such as boundary regression or span generation, while verification is used mainly to rerank candidate segments at inference time. We study whether a frozen verifier can also guide training. Our multi-agent framework couples a trainable \emph{Grounder} with a frozen \emph{Verifier}: the Grounder samples candidate trajectories and evidence segments, the Verifier assigns query-conditioned segment scores, a group-relative policy-gradient objective favors trajectories that outperform their within-input peers, and a bootstrapped calibration loss steers temporal predictions toward verifier-preferred spans. Trained on source tasks and evaluated without target-dataset fine-tuning, a two-billion-parameter instantiation transfers zero-shot across grounded question answering, temporal grounding, and long-video question answering, reaching 28.7\% intersection-over-union and 25.4\% answer-grounding accuracy on a grounded-question-answering benchmark, 46.1\% intersection-over-union on a temporal-grounding benchmark, and 54.1\% on a long-video question-answering benchmark. Relative to a strong same-scale baseline, the gains are modest but consistent, with the clearest improvements on relevance-oriented metrics such as intersection-over-union and moderate-overlap recall. Within the tested benchmarks and transfer setting, the results support frozen verification as a training signal for evidence selection, while showing that strict boundary precision remains comparatively weaker. Code and models are available at https://anonymous.4open.science/r/MASIRL-E50C/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。