基于表示信息改进鲁棒优化,提升跨数据分布的模型泛化能力
Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Framework
- 利用外部表示信息引导鲁棒性扰动方向,仅对关键特征方向增加成本
- 在单细胞多组学数据中,相比传统方法覆盖未来参数更准确
- 适用于多源、多任务迁移学习,可自动调参并构建精准置信区间
分布鲁棒优化(DRO)通过在扰动分布集合上优化最差情况性能来抵御分布偏移。但标准DRO对所有特征扰动一视同仁,可能过度保守。本文提出代表感知的分布鲁棒估计(READ),一种基于Wasserstein距离的DRO框架,利用外部表示信息引导鲁棒性的几何结构。不同于均匀扰动所有协变量方向,READ提高改变表示坐标方向的扰动运输成本,从而将对偶正则化聚焦于表示子空间,同时保留对表示正交方向变化的保护。我们研究了两种场景:一是当前目标的推断,通过渐近分析和Wasserstein轮廓推断法构建与表示对齐的置信区域,并实现自动超参数调优;二是部署至未来群体,若其来自相同的表示不变随机系数模型,所得区域对未来的模型参数具有更高的覆盖率。模拟实验和单细胞多组学应用验证了READ在多源、多任务迁移学习中的优势。
原文摘要 · Abstract (English)
Distributionally robust optimization (DRO) protects statistical learning against distributional shifts by optimizing the worst-case performance over a set of perturbed distributions. However, standard DRO formulations often treat all feature perturbations equally. This can be unnecessarily conservative when external knowledge suggests that the predictive signal is embedded in a low-dimensional representation of covariates. We propose REpresentation-Aware Distributionally robust estimation (READ), a Wasserstein DRO framework that uses external representations to guide the geometry of robustness. Rather than uniformly perturbing all covariate directions, READ increases the transport cost of perturbations that change representation coordinates, thereby reshaping the dual regularization toward the representation subspace. Meanwhile, it preserves protection against variations orthogonal to the representation. We study READ in two regimes. First, for inference on the current target, we characterize our estimator asymptotically and develop a Wasserstein profile inference approach to construct representation-aligned confidence regions while enabling automatic hyperparameter tuning. Second, for deployment to future populations that differ from the current target but are generated from the same representation-invariant random-coefficient model, we show that the resulting regions achieve higher coverage of future model parameters than standard methods. Simulations and a single-cell multi-omics application demonstrate the advantages of READ in multi-source and multitask transfer learning settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。