用简化Transformer建模非线性动态系统，兼具高效与高精度。
Time-Delayed Transformers for Data-Driven Modeling of Low-Dimensional Dynamics
- 单层单头注意力结构，类比时延动态模态分解的非线性推广
- 在混沌系统中准确捕捉长期动态，性能远超传统线性模型
- 参数少、计算快，适合需要可解释性的复杂系统建模
我们提出时间延迟Transformer（TD-TF），一种用于数据驱动建模非定常时空动力学的简化Transformer架构。TD-TF通过表明单层单头Transformer可被解释为时延动态模态分解（TD-DMD）的非线性推广，弥合了基于线性算子的方法与深度序列模型之间的差距。该架构刻意精简，仅包含一个自注意力层（每预测一个查询）和一个前馈层，实现序列长度上的线性计算复杂度和极少参数量。数值实验显示，TD-TF在近线性系统上性能媲美强线性基线，在非线性和混沌区域显著超越它们，能准确捕捉长期动态。在合成信号、非定常空气动力学、Lorenz '63系统及反应-扩散模型上的验证表明，TD-TF在保持线性模型的可解释性与效率的同时，大幅提升了对复杂动态的表达能力。
原文摘要 · Abstract (English)
We propose the time-delayed transformer (TD-TF), a simplified transformer architecture for data-driven modeling of unsteady spatio-temporal dynamics. TD-TF bridges linear operator-based methods and deep sequence models by showing that a single-layer, single-head transformer can be interpreted as a nonlinear generalization of time-delayed dynamic mode decomposition (TD-DMD). The architecture is deliberately minimal, consisting of one self-attention layer with a single query per prediction and one feedforward layer, resulting in linear computational complexity in sequence length and a small parameter count. Numerical experiments demonstrate that TD-TF matches the performance of strong linear baselines on near-linear systems, while significantly outperforming them in nonlinear and chaotic regimes, where it accurately captures long-term dynamics. Validation studies on synthetic signals, unsteady aerodynamics, the Lorenz '63 system, and a reaction-diffusion model show that TD-TF preserves the interpretability and efficiency of linear models while providing substantially enhanced expressive power for complex dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。