揭示时间序列大模型的内部表示结构，实现高效剪枝与概念操控。
Exploring Representations and Interventions in Time Series Foundation Models
- 发现模型层间存在块状冗余，支持智能剪枝提升推理效率。
- 通过潜空间操控可为无周期/趋势信号添加新特征。
- 适合关注模型可解释性与可控生成的研究者。
时间序列基础模型（TSFMs）在众多应用中展现出强大潜力，但其内部表示与学习到的概念仍不清晰。本文研究了多种TSFMs的表示结构与冗余性，分析了不同模型规模下各层间的自相似性。结果揭示出表示中的块状冗余结构，可用于指导剪枝以提升推理速度与效率。此外，我们探索了模型所学概念（如周期性、趋势）及其在潜空间中的操控方式。实验表明，通过潜空间干预可向原本无周期或趋势的信号引入新特征。这些发现凸显了表示分析对模型优化的价值，并展示了概念操控为时间序列分析带来的更高效、可控的新可能。
原文摘要 · Abstract (English)
Time series foundation models (TSFMs) promise to be powerful tools for a wide range of applications. However, their internal representations and learned concepts are still not well understood. In this study, we investigate the structure and redundancy of representations across various TSFMs, examining the self-similarity of model layers within and across different model sizes. This analysis reveals block-like redundancy in the representations, which can be utilized for informed pruning to improve inference speed and efficiency. Additionally, we explore the concepts learned by these models - such as periodicity and trends - and how these can be manipulated through latent space steering to influence model behavior. Our experiments show that steering interventions can introduce new features, e.g., adding periodicity or trends to signals that initially lacked them. These findings underscore the value of representational analysis for optimizing models and demonstrate how conceptual steering offers new possibilities for more controlled and efficient time series analysis with TSFMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。