用语法简化提升代码生成效率,不改功能还更省 token
ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation
- 通过 10 条语法规则精简 Python 代码,保留语义
- 在 HumanEval 上实现 18.1%~37.8% 的生成效率提升
- 适合关注推理效率与模型压缩的开发者
代码生成旨在将用户需求自动转化为可执行代码,显著降低人工开发成本并提升软件生产力。大语言模型(LLMs)的出现推动了代码生成的发展,但其效率仍受架构固有约束影响:每生成一个 token 都需完整推理,持续占用内存并增加资源消耗。现有研究多聚焦于提示压缩和模型量化等推理阶段优化,生成阶段仍被忽视。为此,我们提出知识增强框架 ShortCoder,通过语法优化提升代码生成效率并保持语义等价性与可读性。具体包括:(1) 基于 AST 保持的 10 条 Python 语法简化规则,实现 18.1% 的 token 减少且功能无损;(2) 融合规则重写与 LLM 引导的混合数据合成流程,构建 ShorterCodeBench 数据集,包含经验证的原始代码与简化代码对;(3) 注入简洁性意识的微调策略。实验表明,ShortCoder 在 HumanEval 上持续优于现有方法,生成效率较前人工作提升 18.1%~37.8%,同时保障代码生成性能。
原文摘要 · Abstract (English)
Code generation tasks aim to automate the conversion of user requirements into executable code, significantly reducing manual development efforts and enhancing software productivity. The emergence of large language models (LLMs) has significantly advanced code generation, though their efficiency is still impacted by certain inherent architectural constraints. Each token generation necessitates a complete inference pass, requiring persistent retention of contextual information in memory and escalating resource consumption. While existing research prioritizes inference-phase optimizations such as prompt compression and model quantization, the generation phase remains underexplored. To tackle these challenges, we propose a knowledge-infused framework named ShortCoder, which optimizes code generation efficiency while preserving semantic equivalence and readability. In particular, we introduce: (1) ten syntax-level simplification rules for Python, derived from AST-preserving transformations, achieving 18.1% token reduction without functional compromise; (2) a hybrid data synthesis pipeline integrating rule-based rewriting with LLM-guided refinement, producing ShorterCodeBench, a corpus of validated tuples of original code and simplified code with semantic consistency; (3) a fine-tuning strategy that injects conciseness awareness into the base LLMs. Extensive experimental results demonstrate that ShortCoder consistently outperforms state-of-the-art methods on HumanEval, achieving an improvement of 18.1%-37.8% in generation efficiency over previous methods while ensuring the performance of code generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。