用累积异常值判断长期变化,让模型更高效地适应环境漂移。
Black-Mamba: Biologically-Inspired Leaky Accumulation for Conceptual Knowledge under Distribution Drift

- 基于累积异常值触发记忆更新,实现选择性自适应
- 在多个非平稳数据集上性能优于或持平现有方法
- 适合需要持续推理的动态系统,如金融预测、传感器监测
真实世界中的预测本质上是非平稳的,未来观测的条件分布随时间演变。现有测试时自适应序列模型通过推理过程更新内部状态来应对这一挑战,但其适应机制依赖于即时预测误差或意外程度。这种耦合会将持久性分布漂移与随机波动混淆,导致不必要的更新和低效适应。我们提出Black-Mamba,一种测试时自适应预测架构,将在线适应建模为在分布漂移下受证据控制的状态追踪。该模型在基础预测器基础上引入动态记忆,仅当时间累积的异常值提供充分证据表明存在模式改变时才更新记忆。这使适应变为选择性、事件驱动的过程而非持续行为。在多个具有非平稳动态的预测基准上,Black-Mamba相比现有测试时自适应方法取得竞争性或更优的预测性能,同时显著减少推理期间的记忆更新次数。结合数学分析与生物证据,结果表明累积异常值可作为区分持久漂移与瞬时噪声的合理信号,实现更高效、更鲁棒的适应。
原文摘要 · Abstract (English)
Forecasting under real-world conditions is inherently non-stationary, as the conditional distribution of future observations evolves over time. Recent test-time adaptive sequence models address this challenge by updating internal states during inference, but tie adaptation to instantaneous prediction errors or surprise. This coupling can conflate persistent distribution shift with stochastic innovations, leading to unnecessary updates and inefficient adaptation. We introduce Black-Mamba, a test-time adaptive forecasting architecture that formulates online adaptation as evidence-gated state tracking under distribution drift. The model augments a base predictor with a dynamic memory updated when temporally accumulated surprisal provides sufficient evidence of a regime change. This turns adaptation into a selective, event-driven process rather than a continuous one. Across multiple forecasting benchmarks with non-stationary dynamics, Black-Mamba achieves competitive or improved predictive performance compared to existing test-time adaptation methods while significantly reducing the number of memory updates during inference. Together with mathematical analysis and biological evidence, these results suggest that accumulated surprisal provides a principled signal for distinguishing persistent drift from transient noise, yielding more efficient and robust adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。