提出DEMR框架,提升视频片段检索中对复杂语义的鲁棒性。
Adaptive Evidential Learning for Temporal-Semantic Robustness in Moment Retrieval
- 引入反射翻转融合与查询重建,增强跨模态对齐和文本敏感度
- 通过几何正则化实现不确定性自适应,显著提升难例定位准确率
- 在ActivityNet-CD和Charades-CD上验证了更强的鲁棒性与可解释性
在视频片段检索任务中,基于自然语言查询精准定位时间片段仍具挑战。传统方法依赖预训练模型,难以捕捉细粒度信息且推理确定性高,导致在复杂或模糊片段上对齐困难。为此,本文探索深度证据回归(DER)构建基础证据模型,但发现其存在模态不平衡与启发式不确定性正则化结构差异问题,导致高不确定性错误关联于准确样本而非困难样本。针对此,我们提出去偏证据学习框架DEMR,引入反射翻转融合(RFF)模块加强跨模态对齐,设计查询重建任务提升文本敏感性,减少不确定性估计偏差;同时提出几何正则化(Geom-regularizer)优化不确定性预测,实现对难例的自适应对齐。在标准数据集及去偏数据集ActivityNet-CD、Charades-CD上的大量实验表明,该方法在有效性、鲁棒性和可解释性方面均有显著提升,为时序-语义鲁棒性检索提供新思路。代码已开源:https://github.com/KaijingOfficial/DEMR。
原文摘要 · Abstract (English)
In the domain of moment retrieval, accurately identifying temporal segments within videos based on natural language queries remains challenging. Traditional methods often employ pre-trained models that struggle with fine-grained information and deterministic reasoning, leading to difficulties in aligning with complex or ambiguous moments. To overcome these limitations, we explore Deep Evidential Regression (DER) to construct a vanilla Evidential baseline. However, this approach encounters two major issues: the inability to effectively handle modality imbalance and the structural differences in DER's heuristic uncertainty regularizer, which adversely affect uncertainty estimation. This misalignment results in high uncertainty being incorrectly associated with accurate samples rather than challenging ones. Our observations indicate that existing methods lack the adaptability required for complex video scenarios. In response, we propose Debiased Evidential Learning for Moment Retrieval (DEMR), a novel framework that incorporates a Reflective Flipped Fusion (RFF) block for cross-modal alignment and a query reconstruction task to enhance text sensitivity, thereby reducing bias in uncertainty estimation. Additionally, we introduce a Geom-regularizer to refine uncertainty predictions, enabling adaptive alignment with difficult moments and improving retrieval accuracy. Extensive testing on standard datasets and debiased datasets ActivityNet-CD and Charades-CD demonstrates significant enhancements in effectiveness, robustness, and interpretability, positioning our approach as a promising solution for temporal-semantic robustness in moment retrieval. The code is publicly available at https://github.com/KaijingOfficial/DEMR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。