用图优化算法重构交通预测模型，既轻量又可解释。
Lightweight and Interpretable Transformer via Mixed Graph Algorithm Unrolling for Traffic Forecast
- 将图优化算法迭代展开为前馈网络，替代传统注意力机制。
- 在真实交通数据上达到顶尖预测性能，参数量减少显著。
- 适合需要可解释性与低计算成本的交通预测场景。
不同于传统的“黑箱”Transformer，本文通过展开基于混合图的优化算法，构建了一种轻量且可解释的类Transformer神经网络，用于建模交通数据的时空特征。我们构建两个图：无向图$[0m\mathcal{G}^u$捕捉地理空间相关性，有向图$[0m\mathcal{G}^d$捕捉时间序列依赖关系。假设信号$[0m\mathbf{x}$在两个图上均平滑，设计新的$[0m\ell_2$与$[0m\ell_1$范数变分项以量化并促进有向图上的信号平滑（低频重建）。基于交替方向乘子法（ADMM）设计迭代算法，并将其展开为前馈网络以实现数据驱动参数学习。周期性插入图学习模块以模拟自注意力功能。实验表明，该模型在交通预测任务中性能媲美主流方法，同时大幅降低参数量。
原文摘要 · Abstract (English)
Unlike conventional "black-box" transformers with classical self-attention mechanism, we build a lightweight and interpretable transformer-like neural net by unrolling a mixed-graph-based optimization algorithm to forecast traffic with spatial and temporal dimensions. We construct two graphs: an undirected graph $\mathcal{G}^u$ capturing spatial correlations across geography, and a directed graph $\mathcal{G}^d$ capturing sequential relationships over time. We predict future samples of signal $\mathbf{x}$, assuming it is "smooth" with respect to both $\mathcal{G}^u$ and $\mathcal{G}^d$, where we design new $\ell_2$ and $\ell_1$-norm variational terms to quantify and promote signal smoothness (low-frequency reconstruction) on a directed graph. We design an iterative algorithm based on alternating direction method of multipliers (ADMM), and unroll it into a feed-forward network for data-driven parameter learning. We periodically insert graph learning modules for $\mathcal{G}^u$ and $\mathcal{G}^d$ that play the role of self-attention. Experiments show that our unrolled networks achieve competitive traffic forecast performance as state-of-the-art prediction schemes, while reducing parameter counts drastically.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。