证明多分布学习的随机化算法难被去随机化,但发现一种条件可高效转为确定性模型。
Derandomizing Multi-Distribution Learning
- 通过不等式最小化归约,证明去随机化计算困难
- 即使经验风险最小化高效,去随机化仍难实现
- 提出结构条件,可黑箱转换现有随机模型为确定性模型
多分布学习旨在训练一个在多个数据分布上表现良好的单一预测器,利用每个分布的样本进行训练。近期研究聚焦于二元损失与有限VC维类,已证明近似最优的样本复杂度可通过预言机高效的算法实现,即在类中存在高效经验风险最小化(ERM)的前提下,算法在计算上是高效的。然而,与经典PAC学习中最优样本复杂度由确定性预测器实现不同,当前多分布学习算法输出的是随机化预测器。这引出一个问题:能否将这些算法去随机化以生成适用于多分布的确定性预测器?通过将其归约为不等式最小化问题,我们证明了去随机化在计算上是困难的,即使ERM是高效的。正面结果是,我们识别出一种结构条件,能够实现高效的黑箱归约,将现有的随机多分布预测器转化为确定性版本。
原文摘要 · Abstract (English)
Multi-distribution or collaborative learning involves learning a single predictor that works well across multiple data distributions, using samples from each during training. Recent research on multi-distribution learning, focusing on binary loss and finite VC dimension classes, has shown near-optimal sample complexity that is achieved with oracle efficient algorithms. That is, these algorithms are computationally efficient given an efficient ERM for the class. Unlike in classical PAC learning, where the optimal sample complexity is achieved with deterministic predictors, current multi-distribution learning algorithms output randomized predictors. This raises the question: can these algorithms be derandomized to produce a deterministic predictor for multiple distributions? Through a reduction to discrepancy minimization, we show that derandomizing multi-distribution learning is computationally hard, even when ERM is computationally efficient. On the positive side, we identify a structural condition enabling an efficient black-box reduction, converting existing randomized multi-distribution predictors into deterministic ones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。