arXiv:2502.04593cs.LGcs.AI2025-02被引 7

动态调整噪声水平,让模型更懂复杂时序数据。

The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance

  • 用Vendi分数判断输入是否噪声,动态调节历史依赖强度。
  • 在神经解码和时间预测任务中,性能超越现有模型。
  • 适合处理噪声波动的时序数据,如脑电、传感器信号。

当前最先进的动态模型(如Mamba)假设序列中所有元素具有相同噪声水平,限制了其在噪声时序数据上的表现。本文提出α-Alternator,一种新型生成模型,能动态适应序列中变化的噪声水平。该模型利用Vendi Score(VS)——一种灵活的基于相似性的多样性度量——在每个时间步t,决定当前序列元素与截至该时刻的隐状态对预测未来动态的影响权重。这一权重参数在数据集中所有序列间共享,其符号决定影响方向:负值表示数据嘈杂,提升VS的元素被视为噪声,模型更依赖隐历史;正值则表示该元素信息丰富,模型更重视新输入而非历史。α-Alternator通过观察掩码与Alternator损失最小化联合训练,掩码模拟序列中噪声水平的变化,增强模型鲁棒性。实验表明,α-Alternator在神经解码与时间序列预测基准上均优于Alternators及主流状态空间模型。

原文摘要 · Abstract (English)

Current state-of-the-art dynamical models, such as Mamba, assume the same level of noisiness for all elements of a given sequence, which limits their performance on noisy temporal data. In this paper, we introduce the $α$-Alternator, a novel generative model for time-dependent data that dynamically adapts to the complexity introduced by varying noise levels in sequences. The $α$-Alternator leverages the Vendi Score (VS), a flexible similarity-based diversity metric, to adjust, at each time step $t$, the influence of the sequence element at time $t$ and the latent representation of the dynamics up to that time step on the predicted future dynamics. This influence is captured by a parameter that is learned and shared across all sequences in a given dataset. The sign of this parameter determines the direction of influence. A negative value indicates a noisy dataset, where a sequence element that increases the VS is considered noisy, and the model relies more on the latent history when processing that element. Conversely, when the parameter is positive, a sequence element that increases the VS is considered informative, and the $α$-Alternator relies more on this new input than on the latent history when updating its predicted latent dynamics. The $α$-Alternator is trained using a combination of observation masking and Alternator loss minimization. Masking simulates varying noise levels in sequences, enabling the model to be more robust to these fluctuations and improving its performance in trajectory prediction, imputation, and forecasting. Our experimental results demonstrate that the $α$-Alternator outperforms both Alternators and state-of-the-art state-space models across neural decoding and time-series forecasting benchmarks.

时序建模动态适应噪声鲁棒Vendi Score

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。