将注意力机制与神经动力学结合，实现更可解释的图数据推理。
Controlled Dynamics Attractor Transformer
- 用混合冯·米塞斯-费舍尔注意力能量与霍普菲尔德优化能量耦合建模
- 在多个图分类和异常检测任务上达到当前最优性能
- 引入抑制-兴奋调节机制，提升模型生物合理性与稳定性
Transformer架构通过自注意力机制极大推动了深度模型的表征学习与推理。与此同时，关联记忆（AM）框架将表征映射到能量景观，提供可解释的检索机制。然而，其连续时间推理动态缺乏经典连续吸引子神经网络（CANNs）的生物合理性。为此，我们提出受控动力学吸引子Transformer（CDAT），将混合冯·米塞斯-费舍尔（Mo-vMF）注意力能量与霍普菲尔德精炼能量耦合，并引入受CANN启发的兴奋-抑制调制以增强能量下降。CDAT构建了一个拓扑约束的动力系统，其耦合关系编码了标记间的结构关系，从而将吸引子式动态与现代基于能量的注意力相连接。我们进一步提供构造性耗散分析，正式确立其受控推理动态。得益于这些鲁棒且结构化的动态，CDAT在多个图异常检测和图分类基准上取得当前最优表现。
原文摘要 · Abstract (English)
Transformer architectures have dramatically advanced representation learning and inference in deep models through self-attention mechanisms. In parallel,associative memory (AM) frameworks map representations onto energy landscapes, offering interpretable retrieval mechanisms. However, their continuous-time inference dynamics lack the biological plausibility of classical Continuous Attractor Neural Networks (CANNs). To bridge this gap, we propose Controlled Dynamics Attractor Transformer (CDAT), which couples a mixture von Mises-Fisher (Mo-vMF) attention energy with a Hopfield refinement energy, while augmenting energy descent with a CANN-inspired excitation-inhibition modulation. CDAT instantiates a topology-constrained dynamical system whose couplings encode relational structure among tokens, thereby linking attractor-style dynamics to modern energy-based attention. We further provide a constructive dissipation analysis to formally establish their controlled inference dynamics. Benefiting from these robust and structured dynamics, CDAT achieves state-of-the-art performance across multiple benchmarks in graph anomaly detection and graph classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。