用碳税机制压缩大模型，让编程AI更省资源、更环保。
Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models
- 模仿碳税设计压缩管道，惩罚低效架构，奖励可部署压缩。
- 内存减少49倍，推理速度提升3-10倍，碳排放降低81%。
- 适合关注绿色AI与模型部署效率的开发者和研究者。
大型语言模型在软件工程中的广泛应用带来了不可持续的计算成本危机。尽管这些模型在代码克隆检测、代码摘要和代码生成等任务中表现优异，但其规模庞大、部署缓慢、内存占用高且碳排放量大，威胁了AI赋能软件工程的可扩展性与长期环境可持续性。为此，我们提出碳税变压器（CTT），一种受经济碳定价启发的系统性多架构压缩流水线。通过引入计算碳税，对架构低效进行惩罚，对部署友好压缩给予奖励。我们在三种核心软件工程任务上评估了CTT，涵盖编码器仅用、编码器-解码器和解码器仅用三类模型。结果表明，CTT在推理阶段实现：（1）最高49倍内存缩减，（2）克隆检测速度提升8-10倍，摘要生成提速3倍，代码生成提速4-7倍，（3）最高81%的二氧化碳减排，（4）克隆检测准确率维持约98%，摘要任务达89%，代码生成任务在文本指标上保持91%，通过率（pass@1）达68%。两次消融实验验证了流水线顺序与各组件贡献的重要性，为设计有效性提供实证支持。该工作为软件工程中的负责任人工智能提供了高效且性能不降的可行路径。
原文摘要 · Abstract (English)
The accelerating adoption of Large Language Models (LLMs) in software engineering (SE) has brought with it a silent crisis: unsustainable computational cost. While these models demonstrate remarkable capabilities in different SE tasks, they are unmanageably large, slow to deploy, memory-intensive, and carbon-heavy. This reality threatens not only the scalability and accessibility of AI-powered SE, but also its long-term environmental sustainability. The research challenge is clear: we must go beyond accuracy and address efficiency and environmental cost as first-class design constraints. To meet this challenge, we introduce Carbon-Taxed Transformers (CTT), a systematic multi-architectural compression principled pipeline ordering inspired by economic carbon taxation principles. Drawing from the economic concept of carbon pricing, CTT operationalizes a computational carbon tax that penalizes architectural inefficiencies and rewards deployment-ready compression. We evaluate CTT across three core SE tasks: code clone detection, code summarization, and code generation, with models spanning encoder-only, encoder-decoder, and decoder-only architecture. Our results show that CTT delivers on inference: (1) up to 49x memory reduction, (2) time reduction up to 8-10x for clone detection, up to 3x for summarization, and 4-7x for generation, (3) up to 81% reduction in CO2 emissions and (4) CTT retains around 98% accuracy on clone detection, around 89% on summarization, and up to 91% (textual metrics) and 68% (pass@1) for generation. Two ablation studies show that pipeline ordering and individual component contributions are both essential, providing empirical justification for CTT's design and effectiveness. This work establishes a viable path toward responsible AI in SE through aggressive yet performance-preserving compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。