让Transformer神经过程支持实时数据流更新,速度提升数十倍。
Incremental Transformer Neural Processes
- 用因果掩码和键值缓存实现增量更新,计算复杂度从二次降到线性。
- 在表格回归与温度预测任务中性能接近甚至超过传统模型。
- 适合需要持续推理的场景,如传感器监控、实时数据库分析。
神经过程(NPs)及特定的Transformer神经过程(TNPs)在时空预测到表格数据建模等任务中表现出色。然而,许多应用场景具有序列特性,如实时传感器读数或数据库更新,要求模型能以低成本进行增量更新,而非每次新观测都重新计算内部表示——现有TNP变体缺乏此能力。受大语言模型启发,本文提出增量TNP(incTNP),通过因果掩码、键值(KV)缓存和数据高效的自回归训练策略,在保持标准TNP预测性能的同时,将更新计算成本从二次时间复杂度降至线性。我们在一系列合成与真实世界任务上评估该模型,包括表格回归与温度预测。结果表明,令人惊讶的是,incTNP在性能上可媲美甚至优于非因果TNP,同时在顺序推断中实现数量级的速度提升。最后,我们通过‘隐式贝叶斯性’度量评估模型更新一致性,在逐条流式输入下,incTNP的预测规则仍具有与标准非因果TNP相当的隐式贝叶斯特性,证明其在获得计算优势的同时,未牺牲流式推理所需的稳定性。
原文摘要 · Abstract (English)
Neural Processes (NPs), and specifically Transformer Neural Processes (TNPs), have demonstrated remarkable performance across tasks ranging from spatiotemporal forecasting to tabular data modelling. However, many of these applications are inherently sequential, involving continuous data streams such as real-time sensor readings or database updates. In such settings, models should support cheap, incremental updates rather than recomputing internal representations from scratch for every new observation -- a capability existing TNP variants lack. Drawing inspiration from Large Language Models, we introduce the Incremental TNP (incTNP). By leveraging causal masking, Key-Value (KV) caching, and a data-efficient autoregressive training strategy, incTNP matches the predictive performance of standard TNPs while reducing the computational cost of updates from quadratic to linear time complexity. We empirically evaluate our model on a range of synthetic and real-world tasks, including tabular regression and temperature prediction. Our results show that, surprisingly, incTNP delivers performance comparable to -- or better than -- non-causal TNPs while unlocking orders-of-magnitude speedups for sequential inference. Finally, we assess the consistency of the model's updates -- by adapting a metric of "implicit Bayesianness", we show that under a one-at-a-time streaming protocol, incTNP retains a prediction rule as implicitly Bayesian as standard non-causal TNPs, demonstrating that incTNP achieves the computational benefits of causal masking without sacrificing the consistency required for streaming inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。