arXiv:2411.10077cs.CV2024-11被引 1

通过多视角组合蒸馏,提升图像分类在复杂场景下的准确率与灵活性。

Hierarchical mutual distillation for multi-view fusion: Learning from all possible view combinations

  • 利用所有可能的视角组合进行预测,并通过分层互蒸馏增强一致性。
  • 在多个大规模数据集上达到当前最优分类精度,且支持灵活的视角数量变化。
  • 支持训练与推理视角数量不一致,适合真实世界中视角缺失或变化的场景。

多视角学习常难以有效利用来自不同角度和位置的图像。针对非结构化多视角图像的学习方法仍较少被研究。本文提出一种新型分层互蒸馏多视角融合方法(HMDMV),可同时处理结构化与非结构化多视角场景。该方法利用所有可能的视角组合——单视角、部分多视角、全多视角进行预测,并通过分层互蒸馏提升视角间的一致性。引入基于不确定性的加权机制,根据每个视角组合的预测置信度动态调整其影响,降低低置信度视角的干扰。在大规模结构化与非结构化数据集上的大量实验表明,HMDMV始终达到最先进的分类准确率。另一个独特优势是推理时可灵活调整视角数量,无需额外处理。此外,我们设计了轻量版本,通过每轮训练随机采样视角组合子集,显著降低训练成本。结果表明,HMDMV在视角可用性变化或不完整的现实场景中具备强鲁棒性。代码已开源:https://github.com/labhai/HMDMV。

原文摘要 · Abstract (English)

Multi-view learning often struggles to effectively leverage images captured from diverse angles and locations. Learning methods for unstructured multi-view images remain largely underexplored. We propose a novel Hierarchical Mutual Distillation for Multi-View Fusion (HMDMV) method, which can handle both structured and unstructured multi-view scenarios. It makes predictions utilizing all possible view combinations: single view, partial multi-view, and full multi-view. The method generates predictions for each view combination and then applies hierarchical mutual distillation to enhance inter-view consistency. An uncertainty-based weighting mechanism further refines the fusion process by adjusting the influence of each view combination according to its prediction confidence, reducing the impact of low-confidence views. Extensive experiments on large-scale structured and unstructured datasets demonstrate that HMDMV consistently achieves state-of-the-art classification accuracy. Another unique advantage of HMDMV is that it provides improved flexibility in inference, allowing for more or fewer view counts in inference than those used in training without additional processing. We also provide a light version with reduced training cost by designing an efficient strategy that randomly samples subsets of view combinations during each training iteration. These results highlight HMDMV's robustness in real-world settings where view availability is variable or incomplete. The code is available at https://github.com/labhai/HMDMV.

多视角学习图像分类蒸馏方法灵活性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。