通过智能筛选与融合视角，提升动作能力评估精度
Moving Beyond More Views: Redundancy-Aware Ego-Exo Fusion for Proficiency Estimation
- 自适应选择关键视角，减少冗余信息干扰
- 在两个数据集上达到新最佳性能，准确率显著提升
- 适合需要多视角动作分析的智能评估系统使用
EgoExo 动作能力评估旨在融合第一人称视角的细微动作线索与第三人称视角的空间上下文信息。简单增加第三人称视角会降低性能，因冗余或噪声视角会稀释有效运动线索。分析发现两大原因：(1) 多视角冗余——某些视角提供有限或噪声信息，削弱判别性特征；(2) 过拟合——传统融合增加表征复杂度，模型易记忆特定视角模式而非学习通用表示。为此，我们提出两个互补模块：AdaMVS 在数据层面通过弱监督自适应识别并融合最具信息量的视角标记；VIB-GB 结合梯度混合与变分信息瓶颈正则化，在特征层面压缩冗余信号并抑制训练过拟合。在 EgoExo-4D 与 EgoExo-Fitness 数据集上的实验表明，本方法不仅学会看哪个视角，还知道如何融合，达到新最优结果。源码已公开于 https://github.com/dx199771/AdaMVS。
原文摘要 · Abstract (English)
EgoExo proficiency estimation aims to assess action quality by integrating fine-grained motion cues from egocentric (1st-person) views with spatial context from multiple exocentric (3rd-person) views. Simply adding more exocentric views degrades EgoExo performance, as redundant or noisy perspectives dilute useful motion cues. Our analysis identifies two key causes: (1) Multiview redundancy - From the data perspective, certain views provide limited or noisy information, diluting discriminative cues; (2) Overfitting - From the feature perspective, conventional fusion increases representational complexity, causing the model to memorise view-specific patterns rather than learn generalisable representations. To address these issues, we propose two complementary modules: AdaMVS, which adaptively identifies and fuses the most informative view tokens under weak supervision from the data perspective, and VIB-GB, which combines Gradient Blending and Variational Information Bottleneck regularisation from the feature perspective to compress redundant signals and suppress overfitting during training. Experiments on EgoExo-4D and EgoExo-Fitness demonstrate that our method learns both which view to look at and how to fuse them, achieving new state-of-the-art results. Our source code is available at https://github.com/dx199771/AdaMVS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。