提出新框架解决迁移学习中因分布偏移导致的灾难性负迁移问题。
Le Cam Distortion: A Decision-Theoretic Framework for Robust Transfer Learning
- 基于莱卡姆统计理论,用方向性可模拟替代对称不变性
- 在基因组、图像分类和强化学习中实现零源性能损失
- 适合医疗影像、自动驾驶等对安全要求极高的场景
分布偏移是现实机器学习的核心挑战。主流无监督域适应(UDA)方法通过最小化对称散度强制特征不变性,但当源域与目标域信息量不同时,严格不变性会破坏信息,引发灾难性负迁移。本文提出基于莱卡姆统计实验理论的决策论框架,以构造性近似取代对称不变性,引入莱卡姆失真(Le Cam Distortion),其缺陷距离 $δ(E_1, E_2)$ 可作为条件转移风险的严格上界。该框架通过学习一个核函数,将源域模拟为目标域,实现无需牺牲源域性能的迁移。五个实验(基因组学、视觉、强化学习)验证:(1)人类白细胞抗原基因分型中频率估计相关性达 $r=0.999$,媲美经典方法;(2)CIFAR-10图像分类中源模型准确率保持81.2%,而CycleGAN方法下降至34.7%;(3)强化学习控制任务中避免了基于不变性的方法所出现的策略崩溃。该框架为医疗影像、自主系统、精准医学等不可接受负迁移的场景提供了首个风险可控的迁移学习原理基础。
原文摘要 · Abstract (English)
Distribution shift is the defining challenge of real-world machine learning. The dominant paradigm--Unsupervised Domain Adaptation (UDA)--enforces feature invariance, aligning source and target representations via symmetric divergence minimization [Ganin et al., 2016]. We demonstrate that this approach is fundamentally flawed: when domains are unequally informative (e.g., high-quality vs degraded sensors), strict invariance necessitates information destruction, causing "negative transfer" that can be catastrophic in safety-critical applications [Wang et al., 2019]. We propose a decision-theoretic framework grounded in Le Cam's theory of statistical experiments [Le Cam, 1986], using constructive approximations to replace symmetric invariance with directional simulability. We introduce Le Cam Distortion, quantified by the Deficiency Distance $δ(E_1, E_2)$, as a rigorous upper bound for transfer risk conditional on simulability. Our framework enables transfer without source degradation by learning a kernel that simulates the target from the source. Across five experiments (genomics, vision, reinforcement learning), Le Cam Distortion achieves: (1) near-perfect frequency estimation in HLA genomics (correlation $r=0.999$, matching classical methods), (2) zero source utility loss in CIFAR-10 image classification (81.2% accuracy preserved vs 34.7% drop for CycleGAN), and (3) safe policy transfer in RL control where invariance-based methods suffer catastrophic collapse. Le Cam Distortion provides the first principled framework for risk-controlled transfer learning in domains where negative transfer is unacceptable: medical imaging, autonomous systems, and precision medicine.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。