让模型自动适应数据分布变化，提升在线学习稳定性。
Adapting to Online Distribution Shifts in Deep Learning: A Black-Box Approach
- 用多分辨率实例与时间交叉验证，动态调整学习窗口。
- 仅需维护O(log T)个学习器，计算高效且适应性强。
- 适用于文本、图像等多模态任务，显著提升分类准确率。
我们研究在线分布漂移问题：数据以批次形式到达，每批数据的分布可能随时间任意变化。由于漂移幅度和速率不一，所需的历史数据长度会动态变化，给算法设计带来挑战——既要自适应最佳‘关注窗口’，又要保持计算效率。我们提出一种元算法，输入任意网络架构和任意在线学习（OL）算法，即可生成新算法，在非平稳环境下可证明提升原OL的性能。该算法高效（仅需维护O(log T)个OL实例），并能自动在每个时间点选择最优‘注意力’长度。在多个真实世界数据集上的实验表明，该方法在文本与图像模态的任务中，持续提升用户指定的OL算法的分类精度。关键创新包括受小波理论启发的‘多分辨率实例’设计，以及基于时间的交叉验证技术，二者均可独立应用。
原文摘要 · Abstract (English)
We study the well-motivated problem of online distribution shift in which the data arrive in batches and the distribution of each batch can change arbitrarily over time. Since the shifts can be large or small, abrupt or gradual, the length of the relevant historical data to learn from may vary over time, which poses a major challenge in designing algorithms that can automatically adapt to the best ``attention span'' while remaining computationally efficient. We propose a meta-algorithm that takes any network architecture and any Online Learner (OL) algorithm as input and produces a new algorithm which provably enhances the performance of the given OL under non-stationarity. Our algorithm is efficient (it requires maintaining only $O(\log(T))$ OL instances) and adaptive (it automatically chooses OL instances with the ideal ``attention'' length at every timestamp). Experiments on various real-world datasets across text and image modalities show that our method consistently improves the accuracy of user specified OL algorithms for classification tasks. Key novel algorithmic ingredients include a \emph{multi-resolution instance} design inspired by wavelet theory and a cross-validation-through-time technique. Both could be of independent interest.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。