针对时序语言模型，提出非均匀令牌压缩方法，提升推理速度与精度。
Beyond Uniform Tokens: Adaptive Compression for Time Series Language Models
- 基于频域结构动态压缩时序令牌，减少冗余信息。
- 逐层渐进削减提示令牌，避免深层冗余，实现7.68倍加速。
- 适用于时序预测、分类等多任务，适合构建高效时序大模型。
大型语言模型（LLMs）通过共享标记接口联合建模数值观测与文本上下文，实现了时序（TS）分析。然而，时序标记与提示标记具有本质不同的信息结构，统一处理方式效率低下。本文从非对称标记视角研究时序语言建模中的标记效率问题。发现时序标记的频谱贡献极不均匀：多数标记存在重复频率模式，而少数关键标记保留了重要时间证据。同时观察到提示标记的影响随模型深度衰减，表明全层保留提示标记并非必要。基于此，我们提出一种自适应标记预算框架，通过频域结构压缩时序标记，并在各层逐步减少提示标记。在预测、分类、缺失值填补和异常检测任务上的实验表明，该方法实现最高达7.68×的推理加速，在78%的测试场景中获得性能提升，验证了非对称标记压缩在可扩展时序基础模型中的有效性。
原文摘要 · Abstract (English)
Large language models (LLMs) have enabled time series (TS) analysis by jointly modeling numerical observations and textual context through a shared token interface. However, TS tokens and prompt tokens exhibit fundamentally different information structures, making uniform token processing inefficient. In this paper, we study token efficiency in TS language modeling from an asymmetric-token perspective. We show that TS tokens have highly uneven spectral contributions, where many tokens share redundant frequency patterns while a small subset preserves critical temporal evidence. We also observe that prompt-token influence attenuates with model depth, suggesting that full prompt retention across all layers is unnecessary. Based on these findings, we develop an adaptive token budgeting framework that compresses TS tokens via frequency-domain structure and progressively reduces prompt tokens across layers. Experiments across forecasting, classification, imputation, and anomaly detection demonstrate up to \textit{\textbf{7.68$\times$}} inference acceleration and performance gains in \textit{\textbf{78\%}} of evaluated settings, showing the effectiveness of asymmetric token compression for scalable TS foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。