让分类模型主动检索历史数据，提升分布漂移下的鲁棒性。
Learning to Query History: Nonstationary Classification via Learned Retrieval
- 通过可学习的查询机制从历史数据中动态检索相关样本
- 在亚马逊电子商品评论数据上，对分布漂移的抗性显著优于传统分类器
- 支持大规模历史数据存储，显存占用随历史长度线性增长
非平稳性在实际分类任务中普遍存在，导致部署后的模型性能下降，即使其在训练时的预留数据集上表现良好。本文将非平稳分类重构为时间序列预测问题：分类不仅依赖当前输入，还依赖超过训练截止点的历史标注样本序列。为应对长序列问题，提出一种可学习的离散检索机制，通过输入相关的查询从历史中采样相关样本，并与分类器端到端联合训练，使用基于得分的梯度估计器。该方法使历史数据可在任意文件系统中存储，无需加载至内存。在合成基准和 Amazon Reviews '23（电子品类）数据集上的实验表明，相比标准分类器，该方法在分布漂移下表现出更强的鲁棒性，且显存占用随历史序列长度可预测地增长。
原文摘要 · Abstract (English)
Nonstationarity is ubiquitous in practical classification settings, leading deployed models to perform poorly even when they generalize well to holdout sets available at training time. We address this by reframing nonstationary classification as time series prediction: rather than predicting from the current input alone, we condition the classifier on a sequence of historical labeled examples that extends beyond the training cutoff. To scale to large sequences, we introduce a learned discrete retrieval mechanism that samples relevant historical examples via input-dependent queries, trained end-to-end with the classifier using a score-based gradient estimator. This enables the full corpus of historical data to remain on an arbitrary filesystem during training and deployment. Experiments on synthetic benchmarks and Amazon Reviews '23 (electronics category) show improved robustness to distribution shift compared to standard classifiers, with VRAM scaling predictably as the length of the historical data sequence increases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。