arXiv:2508.04924cs.CV2025-08

提出动态适配测试视频的亮点检测方法,提升模型泛化能力。

Test-Time Adaptation for Video Highlight Detection Using Meta-Auxiliary Learning and Cross-Modality Hallucinations

  • 测试时通过辅助任务动态调整模型,适应不同视频特征
  • 在三个数据集上提升现有模型性能,显著改善检测准确率
  • 适合需要高适应性的视频分析场景,如跨平台内容推荐

现有视频亮点检测方法虽先进,但在泛化到新测试视频时表现不佳。这些方法通常对每个测试视频使用通用模型,无法捕捉个别视频的内容、风格或音视频质量差异,导致性能下降。本文提出Highlight-TTA框架,通过测试时自适应(Test-Time Adaptation)动态调整模型,使其更贴合测试视频特性,从而提升泛化与检测效果。该框架联合优化主任务(亮点检测)与辅助任务(跨模态幻觉生成),采用元辅助训练策略,在训练阶段增强适应能力,并在测试阶段利用辅助任务进一步优化模型。在三个前沿亮点检测模型和三个基准数据集上的实验表明,引入Highlight-TTA后,各模型性能均显著提升,达到更优结果。

原文摘要 · Abstract (English)

Existing video highlight detection methods, although advanced, struggle to generalize well to all test videos. These methods typically employ a generic highlight detection model for each test video, which is suboptimal as it fails to account for the unique characteristics and variations of individual test videos. Such fixed models do not adapt to the diverse content, styles, or audio and visual qualities present in new, unseen test videos, leading to reduced highlight detection performance. In this paper, we propose Highlight-TTA, a test-time adaptation framework for video highlight detection that addresses this limitation by dynamically adapting the model during testing to better align with the specific characteristics of each test video, thereby improving generalization and highlight detection performance. Highlight-TTA is jointly optimized with an auxiliary task, cross-modality hallucinations, alongside the primary highlight detection task. We utilize a meta-auxiliary training scheme to enable effective adaptation through the auxiliary task while enhancing the primary task. During testing, we adapt the trained model using the auxiliary task on the test video to further enhance its highlight detection performance. Extensive experiments with three state-of-the-art highlight detection models and three benchmark datasets show that the introduction of Highlight-TTA to these models improves their performance, yielding superior results.

视频亮点检测测试时自适应跨模态生成动态调整

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。