SIDDA通过自动对齐域分布,显著提升模型在不同数据下的分类准确率和校准性。
SIDDA: SInkhorn Dynamic Domain Adaptation for Image Classification with Equivariant Neural Networks
- 基于Sinkhorn散度的自动化域对齐方法,几乎无需调参
- 在无标签目标数据上提升约40%分类准确率,且改善模型校准性
- 特别适合与等变神经网络结合,适用于天文等复杂真实数据
现代神经网络在协变量偏移(训练与测试数据分布不同但标签条件分布不变)下泛化能力差,归因于缺乏域不变特征。现有域自适应(DA)方法需大量超参数调优,计算开销大。本文提出SIDDA,一种基于Sinkhorn散度的即插即用型DA训练算法,可实现高效域对齐且几乎无需调参。在多种模拟与真实数据集(包括简单形状、手写数字、真实天文观测)上验证有效。该方法兼容多种神经网络架构,尤其在与等变神经网络(ENNs)结合时,显著提升分类准确率与模型校准性:在无标签目标数据上准确率提升约40%。研究还发现,随着二面体群$D_N$的阶数增加,模型性能随之提升。此外,SIDDA使源域与目标域的模型校准性均显著改善,ECE与Brier得分降低一个数量级以上。
原文摘要 · Abstract (English)
Modern neural networks (NNs) often do not generalize well in the presence of a "covariate shift"; that is, in situations where the training and test data distributions differ, but the conditional distribution of classification labels remains unchanged. In such cases, NN generalization can be reduced to a problem of learning more domain-invariant features. Domain adaptation (DA) methods include a range of techniques aimed at achieving this; however, these methods have struggled with the need for extensive hyperparameter tuning, which then incurs significant computational costs. In this work, we introduce SIDDA, an out-of-the-box DA training algorithm built upon the Sinkhorn divergence, that can achieve effective domain alignment with minimal hyperparameter tuning and computational overhead. We demonstrate the efficacy of our method on multiple simulated and real datasets of varying complexity, including simple shapes, handwritten digits, and real astronomical observations. SIDDA is compatible with a variety of NN architectures, and it works particularly well in improving classification accuracy and model calibration when paired with equivariant neural networks (ENNs). We find that SIDDA enhances the generalization capabilities of NNs, achieving up to a $\approx40\%$ improvement in classification accuracy on unlabeled target data. We also study the efficacy of DA on ENNs with respect to the varying group orders of the dihedral group $D_N$, and find that the model performance improves as the degree of equivariance increases. Finally, we find that SIDDA enhances model calibration on both source and target data--achieving over an order of magnitude improvement in the ECE and Brier score. SIDDA's versatility, combined with its automated approach to domain alignment, has the potential to advance multi-dataset studies by enabling the development of highly generalizable models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。