首个可证明高效的神经网络分布偏移学习算法，适用于回归任务。
Learning Neural Networks with Distribution Shift: Efficiently Certifiable Guarantees
- 引入数据依赖特征映射与跨分布核矩阵，结合经典核方法解决非凸回归问题。
- 在训练分布尾部为严格亚指数时，可保证高效学习；对有界超收缩分布给出多项式时间算法。
- 首次将分布偏移检测框架拓展至回归场景，适合关注模型鲁棒性与可验证性的研究者。
我们提出了首个在分布偏移下可证明高效的神经网络学习算法。基于Klivans等人（2024）提出的可测试分布偏移学习框架（TDS learning），学习者从训练分布获取带标签样本，从测试分布获取无标签样本，需输出低测试误差的假设，或在检测到分布偏移时拒绝。对测试分布无任何假设。此前所有TDS学习工作仅限分类任务，而本文处理非凸回归情形。结果适用于任意Lipschitz激活函数的实值网络，且当训练分布具有严格亚指数尾部时成立。对于有界且超收缩的训练分布，我们给出了一个针对单隐层网络（含Sigmoid激活）的全多项式时间算法。该成果通过引入数据依赖特征映射和一种耦合训练与测试样本的核矩阵，将经典核方法融入TDS框架。
原文摘要 · Abstract (English)
We give the first provably efficient algorithms for learning neural networks with distribution shift. We work in the Testable Learning with Distribution Shift framework (TDS learning) of Klivans et al. (2024), where the learner receives labeled examples from a training distribution and unlabeled examples from a test distribution and must either output a hypothesis with low test error or reject if distribution shift is detected. No assumptions are made on the test distribution. All prior work in TDS learning focuses on classification, while here we must handle the setting of nonconvex regression. Our results apply to real-valued networks with arbitrary Lipschitz activations and work whenever the training distribution has strictly sub-exponential tails. For training distributions that are bounded and hypercontractive, we give a fully polynomial-time algorithm for TDS learning one hidden-layer networks with sigmoid activations. We achieve this by importing classical kernel methods into the TDS framework using data-dependent feature maps and a type of kernel matrix that couples samples from both train and test distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。