提出首个面向流数据的MMD与CORAL损失方差减少方法
Online Variance Reduction for Domain Adaptation on Streaming Data
- 通过动态更新参考统计量并自适应重加权小批量数据
- 在流数据上实现与离线算法相当的方差降低和准确率
- 适合在线学习、分布式场景，计算开销低
本文研究了最大均值差异（MMD）和相关对齐（CORAL）损失函数的随机方差减少（SVR）问题。尽管已有多种针对这些损失的离线SVR算法，但它们不适用于在线、分布式或增量学习场景。本文提出自适应方差减少的在线重加权方法（ARROW），是首个适用于流数据的MMD与CORAL的在线SVR算法。该方法维护对齐统计量的移动平均参考值，并自适应地重加权传入的小批量数据，使小批量与参考统计量对齐。此外，我们设计了一种松弛重加权方案，使得后续的权重优化问题可解。实验与模拟表明，ARROW在运行时间、方差降低程度和目标域准确率方面均与离线算法表现相当。
原文摘要 · Abstract (English)
This paper studies the problem of stochastic variance reduction (SVR) for the maximum mean discrepancy (MMD) and correlation alignment (CORAL) loss functions. Although various offline SVR algorithms for these losses have been proposed, these are incompatible with online, distributed, or incremental learning settings. This paper presents Adaptive vaRiance Reduction via Online reWeighting (ARROW), the first online SVR algorithm for the MMD and CORAL for streamed data. The method maintains moving average references of the alignment statistics, and adaptively reweights incoming minibatches so that the minibatch and reference statistics are aligned. Further, we propose a relaxed reweighting scheme so that the ensuing weight-optimisation problem is tractable. In experiments and simulations, we show that ARROW performs competitively with offline algorithms in terms of runtime, degree of variance reduction achieved, and target domain accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。