arXiv:2412.07980cs.CVcs.AI2024-12ICLR被引 2

用几何结构提升测试时自适应性能,解决模型部署中的分布偏移问题。

TTVD: Towards a Geometric Framework for Test-Time Adaptation Based on Voronoi Diagram

  • 基于沃罗诺伊图的几何框架,利用空间分区思想优化特征对齐。
  • 在CIFAR-10-C等基准上显著优于现有方法,性能提升明显。
  • 适用于实际场景中的小批量和类别不平衡问题,鲁棒性强。

深度学习模型在真实数据上部署时常因分布偏移而泛化能力下降。测试时自适应(TTA)在推理阶段在线调整模型以缓解此问题。近年来,基于邻近的方案受到关注,其原型嵌入可提供位置信息以缓解训练与测试数据间的特征偏移。然而,这类方法因结构简单,难以捕捉有效模式,常导致性能下降。本文从几何视角重新审视TTA问题,发现其底层结构与经典的沃罗诺伊图(Voronoi Diagram)一致。基于此,提出测试时沃罗诺伊图引导调整(TTVD)框架:1)聚类诱导沃罗诺伊图(CIVD),融合自监督与熵引导方法,增强信息表达;2)加权图(PD),沃罗诺伊图的推广形式,通过为每个胞腔赋予权重细化分区。在CIFAR-10-C、CIFAR-100-C、ImageNet-C和ImageNet-R等严格设置下的实验表明,TTVD显著优于当前最优方法。此外,大量实验分析了批量大小和类别不平衡的影响,验证了框架在真实场景中的鲁棒性与适应性。

原文摘要 · Abstract (English)

Deep learning models often struggle with generalization when deploying on real-world data, due to the common distributional shift to the training data. Test-time adaptation (TTA) is an emerging scheme used at inference time to address this issue. In TTA, models are adapted online at the same time when making predictions to test data. Neighbor-based approaches have gained attention recently, where prototype embeddings provide location information to alleviate the feature shift between training and testing data. However, due to their inherit limitation of simplicity, they often struggle to learn useful patterns and encounter performance degradation. To confront this challenge, we study the TTA problem from a geometric point of view. We first reveal that the underlying structure of neighbor-based methods aligns with the Voronoi Diagram, a classical computational geometry model for space partitioning. Building on this observation, we propose the Test-Time adjustment by Voronoi Diagram guidance (TTVD), a novel framework that leverages the benefits of this geometric property. Specifically, we explore two key structures: 1) Cluster-induced Voronoi Diagram (CIVD): This integrates the joint contribution of self-supervision and entropy-based methods to provide richer information. 2) Power Diagram (PD): A generalized version of the Voronoi Diagram that refines partitions by assigning weights to each Voronoi cell. Our experiments under rigid, peer-reviewed settings on CIFAR-10-C, CIFAR-100-C, ImageNet-C, and ImageNet-R shows that TTVD achieves remarkable improvements compared to state-of-the-art methods. Moreover, extensive experimental results also explore the effects of batch size and class imbalance, which are two scenarios commonly encountered in real-world applications. These analyses further validate the robustness and adaptability of our proposed framework.

测试时自适应几何建模沃罗诺伊图分布外泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。