提出一种渐进域适应方法,能有效控制错误传播。
Gradual Domain Adaptation via Manifold-Constrained Distributionally Robust Optimization
- 基于分布鲁棒优化,自适应调整 Wasserstein 半径
- 理论证明误差可线性或完全消除,依赖于数据流形约束
- 适合处理连续分布变化的场景,如时间序列迁移
本文针对一类流形约束数据分布下的渐进域适应问题。考虑一个包含 $T\ge2$ 个数据分布 $P_1,\ldots,P_T$ 的序列,其中相邻分布 $P_i,P_{i+1}$ 在 Wasserstein 距离下接近。我们有一个大小为 $n$ 的监督数据集来自 $P_0$,后续分布仅提供无标签 i.i.d. 样本。假设所有分布具有已知良好属性(如类内软/硬间隔)。提出一种基于分布鲁棒优化(DRO)的自适应 Wasserstein 半径方法。理论上证明该方法可对所有 $P_i$ 的分类误差进行合理上界控制。边界依赖于新提出的 { t {compatibility}} 度量,完整刻画了误差沿序列传播的动态。对于约束不足的分布,误差会随渐进转移指数级增长;而对于适当约束的分布,误差可保持线性甚至完全消除。实验验证了理论结果。
原文摘要 · Abstract (English)
The aim of this paper is to address the challenge of gradual domain adaptation within a class of manifold-constrained data distributions. In particular, we consider a sequence of $T\ge2$ data distributions $P_1,\ldots,P_T$ undergoing a gradual shift, where each pair of consecutive measures $P_i,P_{i+1}$ are close to each other in Wasserstein distance. We have a supervised dataset of size $n$ sampled from $P_0$, while for the subsequent distributions in the sequence, only unlabeled i.i.d. samples are available. Moreover, we assume that all distributions exhibit a known favorable attribute, such as (but not limited to) having intra-class soft/hard margins. In this context, we propose a methodology rooted in Distributionally Robust Optimization (DRO) with an adaptive Wasserstein radius. We theoretically show that this method guarantees the classification error across all $P_i$s can be suitably bounded. Our bounds rely on a newly introduced {\it {compatibility}} measure, which fully characterizes the error propagation dynamics along the sequence. Specifically, for inadequately constrained distributions, the error can exponentially escalate as we progress through the gradual shifts. Conversely, for appropriately constrained distributions, the error can be demonstrated to be linear or even entirely eradicated. We have substantiated our theoretical findings through several experimental results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。