用少量标签数据实现精准领域迁移,关键在多起点微调与因果干预建模。
When few labeled target data suffice: a theory of semi-supervised domain adaptation via fine-tuning from multiple adaptive starts
- 基于因果模型构建多起点微调框架,利用源数据和无标签目标数据恢复共享结构。
- 仅需少量目标标签即可达到最优性能,样本复杂度与干预维度相关而非全维。
- 适合标签稀缺但有大量源数据和无标签目标数据的场景,如医疗影像迁移。
半监督领域自适应(SSDA)旨在利用丰富的源域数据和未标记的目标域数据,在仅有少量目标标签的情况下实现准确预测。本文在结构因果模型(SCMs)下研究该问题,将源域与目标域之间的分布偏移建模为数据生成过程中的干预,而非参数的随意调整。核心机制是:在低维干预条件下,源数据与未标记目标数据可识别高维共享结构,仅剩低维目标特异性修正需由有限标签数据学习。我们针对三种典型干预模型形式化该原理,并提出FT-DIP、FT-OLS-Src与FT-CIP三种方法。在每种模型下,证明将无监督域适应方法扩展至半监督场景,可在少量目标标签下实现极小化最大误差,且标签样本复杂度依赖于干预维度而非整体维度。当分布偏移不明确时,提出多自适应起点微调(MASFT)算法,通过多个起点微调并用小量目标验证集选择最优解,计算开销仅对数增长。模拟与真实数据实验验证了方法有效性。
原文摘要 · Abstract (English)
Semi-supervised domain adaptation (SSDA) seeks to achieve accurate predictions in a target domain with limited labeled target data by exploiting abundant source and unlabeled target data. We study this problem under structural causal models (SCMs), which provide a statistical framework to describe distribution shifts between source and target domains as interventions in the data-generating process rather than ad hoc changes in model parameters. The central phenomenon is that, under low-dimensional interventions, source and unlabeled target data can help identify the high-dimensional shared structure, leaving only a low-dimensional target-specific correction to be learned from limited labeled target data. We formalize this principle for three canonical intervention models and propose the corresponding SSDA methods FT-DIP, FT-OLS-Src and FT-CIP. Under each intervention model, we demonstrate how extending an unsupervised domain adaptation (UDA) method to SSDA can achieve minimax-optimal target performance with limited target labels, with the labeled-target sample complexity scaling with the intervention dimension rather than the ambient dimension. When the distribution shift is underspecified, we propose the Multi-Adaptive-Start Fine-Tuning (MASFT) algorithm, which fine-tunes from multiple adaptive starts and selects among them using a small target validation set, incurring only logarithmic overhead in the number of starts. We validate the effectiveness of our proposed methods through simulated and real data experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。