轻量级时序预测模型FaCTR,用结构化设计提升效率与可解释性。
FaCTR: Factorized Channel-Temporal Representation Transformers for Efficient Time Series Forecasting
- 通过低秩分解建模跨通道动态交互,结合门控机制注入时序上下文。
- 参数仅约40万,性能超越50倍参数量的基线模型,在11个基准上达最优。
- 支持自监督预训练,适合需要可解释性的工业级时序任务应用。
尽管Transformer在语言和视觉任务中表现优异,但其复杂架构在时序预测中因信息密度低、跨通道依赖复杂而出现收益递减。本文提出轻量级时空Transformer FaCTR,采用显式结构设计应对这一问题。FaCTR通过可学习门控机制,将低秩因子分解机建模的动态对称跨通道交互注入时序上下文补丁嵌入,并编码静态与动态协变量实现多变量条件建模。尽管模型紧凑,其最大变体仅需约40万参数(平均为同类先进Transformer基线的1/50),在涵盖短时与长时预测的11个公开基准上达到当前最佳性能。此外,其结构化设计支持通过跨通道影响得分实现可解释性,满足真实决策场景需求。最后,FaCTR支持自监督预训练,可作为下游时序任务的紧凑通用基础模型。
原文摘要 · Abstract (English)
While Transformers excel in language and vision-where inputs are semantically rich and exhibit univariate dependency structures-their architectural complexity leads to diminishing returns in time series forecasting. Time series data is characterized by low per-timestep information density and complex dependencies across channels and covariates, requiring conditioning on structured variable interactions. To address this mismatch and overparameterization, we propose FaCTR, a lightweight spatiotemporal Transformer with an explicitly structural design. FaCTR injects dynamic, symmetric cross-channel interactions-modeled via a low-rank Factorization Machine into temporally contextualized patch embeddings through a learnable gating mechanism. It further encodes static and dynamic covariates for multivariate conditioning. Despite its compact design, FaCTR achieves state-of-the-art performance on eleven public forecasting benchmarks spanning both short-term and long-term horizons, with its largest variant using close to only 400K parameters-on average 50x smaller than competitive spatiotemporal transformer baselines. In addition, its structured design enables interpretability through cross-channel influence scores-an essential requirement for real-world decision-making. Finally, FaCTR supports self-supervised pretraining, positioning it as a compact yet versatile foundation for downstream time series tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。