解析时间序列模型设计选择如何隐含影响性能表现
Understanding the Implicit Biases of Design Choices for Time Series Foundation Models
- 通过控制实验分析训练参数对模型行为的影响
- 发现设计选择会引发模型对时间模式和均值回归的隐式偏好
- 揭示复杂偏差交互，适合模型设计与评估研究者参考
时间序列基础模型（TSFMs）是一类具有潜力的通用时序预测工具，但其行为受设计中细微归纳偏置的强烈影响。我们不追求提出新模型或在现有基准上胜出，而是旨在理解训练过程中的各项‘调节旋钮’（如分块大小、嵌入方式、训练目标等）如何影响模型质量。结合理论分析与受控实验，我们识别出多个设计选择，并展示它们如何导致模型在时间行为、几何结构、均值回归强度等方面产生隐式偏置；这些偏置可能直观或极具反直觉，取决于模型和数据特性。我们通过异常值处理的案例研究展示了多个偏置的复杂交互；并讨论了结果对学习‘苦涩教训’及构建高效TSFMs的启示。
原文摘要 · Abstract (English)
Time series foundation models (TSFMs) are a class of potentially powerful, general-purpose tools for time series forecasting and related temporal tasks, but their behavior is strongly shaped by subtle inductive biases in their design. Rather than developing a new model and claiming that it is better than existing TSFMs, e.g., by winning on existing well-established benchmarks, our objective is to understand how the various ``knobs'' of the training process affect model quality. Using a mix of theory and controlled empirical evaluation, we identify several design choices (patch size, embedding choice, training objective, etc.) and show how they lead to implicit biases in fundamental model properties (temporal behavior, geometric structure, how aggressively or not the model regresses to the mean, etc.); and we show how these biases can be intuitive or very counterintuitive, depending on properties of the model and data. We also illustrate in a case study on outlier handling how multiple biases can interact in complex ways; and we discuss implications of our results for learning the bitter lesson and building TSFMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。