用因果模型提升视频语义理解在数据不均衡下的鲁棒性
Causality Model for Semantic Understanding on Videos
- 引入因果建模，挖掘视频中隐藏的因果关系而非仅依赖相关性
- 在长尾分布和扰动分布下，性能下降显著低于传统方法
- 适合关注视频理解鲁棒性和可解释性的研究者
经过十年发展，视频理解已进入瓶颈期，单纯依赖海量数据和复杂架构难以应对所有场景。普遍存在的数据不平衡问题使深度神经网络难以学习真实因果机制，导致在分布偏移（如长尾不平衡、扰动不平衡）时性能大幅下降。为此，研究者开始探索捕捉视频数据中因果模式的新方法。本文聚焦语义视频理解领域，研究因果建模在两项基础任务中的潜力：视频关系检测（VidVRD）与视频问答（VideoQA），旨在通过揭示潜在因果结构提升模型鲁棒性。
原文摘要 · Abstract (English)
After a decade of prosperity, the development of video understanding has reached a critical juncture, where the sole reliance on massive data and complex architectures is no longer a one-size-fits-all solution to all situations. The presence of ubiquitous data imbalance hampers DNNs from effectively learning the underlying causal mechanisms, leading to significant performance drops when encountering distribution shifts, such as long-tail imbalances and perturbed imbalances. This realization has prompted researchers to seek alternative methodologies to capture causal patterns in video data. To tackle these challenges and increase the robustness of DNNs, causal modeling emerged as a principle to discover the true causal patterns behind the observed correlations. This thesis focuses on the domain of semantic video understanding and explores the potential of causal modeling to advance two fundamental tasks: Video Relation Detection (VidVRD) and Video Question Answering (VideoQA).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。