提出新指标评估无监督域适应难度,更真实反映学习挑战。
On the Hardness of Unsupervised Domain Adaptation: Optimal Learners and Information-Theoretic Perspective
- 用后验分布建模真实三元组不确定性,定义最优学习器性能。
- 引入后验目标标签不确定性(PTLU)作为风险下界,量化学习难度。
- 相比旧指标,PTLU更能准确判断域适应任务的难易程度,适合研究者使用。
本文研究在协变量偏移下的无监督域适应(UDA)难度。通过在真实三元组 $(p, q, f)$ 上定义分布 $π$(即一个 UDA 类),其中 $(p, q)$ 为源-目标分布对,$f$ 为分类器,建模学习者面临的不确定性。将学习者性能定义为在真实三元组随机性下对目标域风险的平均值。该设定耦合了源、目标分布与真实分类器,不同于传统最坏情况分析中过度强调罕见困难实例的做法。在此框架下,精确刻画了最优学习者,并由此定义了 UDA 类及观测样本的学习难度。为量化难度,引入信息论量——后验目标标签不确定性(PTLU),及其基于样本的估计量 EPTLU,捕捉目标域预测中的不确定性。证明该量可作为任意学习者的风险下界,建议其可作为评估 UDA 难度的代理指标。通过多个例子展示 PTLU 相较于现有度量在评估学习难度上的优势。
原文摘要 · Abstract (English)
This paper studies the hardness of unsupervised domain adaptation (UDA) under covariate shift. We model the uncertainty that the learner faces by a distribution $π$ in the ground-truth triples $(p, q, f)$ -- which we call a UDA class -- where $(p, q)$ is the source -- target distribution pair and $f$ is the classifier. We define the performance of a learner as the overall target domain risk, averaged over the randomness of the ground-truth triple. This formulation couples the source distribution, the target distribution and the classifier in the ground truth, and deviates from the classical worst-case analyses, which pessimistically emphasize the impact of hard but rare UDA instances. In this formulation, we precisely characterize the optimal learner. The performance of the optimal learner then allows us to define the learning difficulty for the UDA class and for the observed sample. To quantify this difficulty, we introduce an information-theoretic quantity -- Posterior Target Label Uncertainty (PTLU) -- along with its empirical estimate (EPTLU) from the sample , which capture the uncertainty in the prediction for the target domain. Briefly, PTLU is the entropy of the predicted label in the target domain under the posterior distribution of ground-truth classifier given the observed source and target samples. By proving that such a quantity serves to lower-bound the risk of any learner, we suggest that these quantities can be used as proxies for evaluating the hardness of UDA learning. We provide several examples to demonstrate the advantage of PTLU, relative to the existing measures, in evaluating the difficulty of UDA learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。