arXiv:2410.03263cs.LGcs.AI2024-10ICLR被引 14

针对回归任务设计了测试时自适应的新方法,提升模型在未知数据上的表现。

Test-time Adaptation for Regression by Subspace Alignment

  • 通过识别关键特征子空间并加权重要维度实现有效对齐
  • 在真实数据集上显著优于现有基线方法
  • 适合需要快速适应新数据的回归场景

本文研究回归任务中的测试时自适应(TTA),即预训练于源域的回归模型如何利用无标签目标数据适应未知的目标分布。尽管回归是机器学习的基本任务,但现有多数TTA方法针对分类设计,依赖类别预测输出,而回归模型仅输出单一标量值,导致传统特征对齐方法失效。我们发现,由于特征分布在小子空间内,且多数原始维度对输出贡献有限,直接沿用分类方法效果不佳。为此,提出显著子空间对齐(SSA),包含子空间检测与维度加权两部分:子空间检测识别对输出有显著影响的特征子空间,随后在该子空间中进行特征对齐;维度加权提升子空间内对输出更重要的特征维度的权重。实验表明,SSA在多个真实数据集上均优于各类基线方法。

原文摘要 · Abstract (English)

This paper investigates test-time adaptation (TTA) for regression, where a regression model pre-trained in a source domain is adapted to an unknown target distribution with unlabeled target data. Although regression is one of the fundamental tasks in machine learning, most of the existing TTA methods have classification-specific designs, which assume that models output class-categorical predictions, whereas regression models typically output only single scalar values. To enable TTA for regression, we adopt a feature alignment approach, which aligns the feature distributions between the source and target domains to mitigate the domain gap. However, we found that naive feature alignment employed in existing TTA methods for classification is ineffective or even worse for regression because the features are distributed in a small subspace and many of the raw feature dimensions have little significance to the output. For an effective feature alignment in TTA for regression, we propose Significant-subspace Alignment (SSA). SSA consists of two components: subspace detection and dimension weighting. Subspace detection finds the feature subspace that is representative and significant to the output. Then, the feature alignment is performed in the subspace during TTA. Meanwhile, dimension weighting raises the importance of the dimensions of the feature subspace that have greater significance to the output. We experimentally show that SSA outperforms various baselines on real-world datasets.

回归测试时适应特征对齐子空间

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。