优化Transformer模型，让时间序列分类更省电更快。
Energy-Efficient Transformer Inference: Optimization Strategies for Time Series Classification
- 用静态量化和L1剪枝优化模型结构。
- 量化降能耗29.14%，剪枝提速63%且精度损失小。
- 适合边缘设备部署的高效模型设计参考。
Transformer模型在时间序列分类中的计算需求日益增长，亟需高效的优化策略以实现节能部署。本研究系统考察了针对Transformer架构的结构化剪枝与量化方法，基于RefrigerationDevices、ElectricDevices和PLAID三个数据集进行大量实验，评估不同配置下的模型性能与能效表现。结果表明，静态量化可降低29.14%的能耗而保持分类性能，L1剪枝在仅造成轻微精度下降的前提下实现63%的推理速度提升。研究为基于Transformer的时间序列分类模型在资源受限环境中的高效部署提供了重要实践依据。
原文摘要 · Abstract (English)
The increasing computational demands of transformer models in time series classification necessitate effective optimization strategies for energy-efficient deployment. Our study presents a systematic investigation of optimization techniques, focusing on structured pruning and quantization methods for transformer architectures. Through extensive experimentation on three distinct datasets (RefrigerationDevices, ElectricDevices, and PLAID), we quantitatively evaluate model performance and energy efficiency across different transformer configurations. Our experimental results demonstrate that static quantization reduces energy consumption by 29.14% while maintaining classification performance, and L1 pruning achieves a 63% improvement in inference speed with minimal accuracy degradation. Our findings provide valuable insights into the effectiveness of optimization strategies for transformer-based time series classification, establishing a foundation for efficient model deployment in resource-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。