通过本地预处理优化多语言代码提示，降低AI编程助手的输入成本。
Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing
- 在本地用小模型将多语言代码提示转为简洁英文任务格式
- 相比原始提示减少34%-47%的提示词数，总词数降18.8%
- 适合关注推理成本与多语言支持的开发者和工具链设计者
AI辅助编程助手受限于输入令牌成本。原始人类输入的两大问题导致了大量开销：非英语文本的分词效率低，以及对话式提示中的结构熵高。现有方法多为被动压缩已膨胀的上下文或故障后干预。本文提出一种前飞、边缘侧的提示重写中间件，部署在开发者与云端代理之间。本地运行的Llama 3.2（3B）模型执行跨语言翻译、结构化改写为紧凑任务导向格式，并通过正则验证与回退机制确保优化后提示不比原提示更大。在涵盖土耳其语、阿拉伯语、中文及代码混杂规格的OMH-Polyglot多语言编码基准上评估，三种商用LLM后端下，该中间件使提示词数减少34%-47%，总词数最多下降18.8%，同时保持或提升任务准确率。消融实验表明收益主要来自重写阶段而非简单函数名提取。与LLMLingua-2在相同压缩率下对比，本方法在所有后端均取得更优的OckScore表现。结果表明，主动提示优化可显著降低推理成本而不牺牲编码质量。
原文摘要 · Abstract (English)
AI-assisted coding agents are bottlenecked by input-token cost. Two pathologies of raw human input drive much of this overhead: tokenization inefficiency for non-English text and structural entropy in conversational prompts. Existing approaches act reactively by compressing already-bloated contexts or intervening after failures occur. We introduce a pre-flight, edge-side prompt-rewriting middleware that operates between the developer and the cloud agent. A local Llama 3.2 (3B) model performs cross-lingual translation into English, structural rewriting into a compact task-oriented format, and regex-validated rewrite-with-fallback safeguards to ensure the optimized prompt is never larger than the original. We evaluate on OMH-Polyglot, a multilingual coding benchmark spanning Turkish, Arabic, Chinese, and code-switched specifications. Across three commercial LLM backends, the middleware reduces prompt tokens by 34-47 percent and total tokens by up to 18.8 percent while preserving or improving task accuracy. Ablation studies show that gains arise primarily from the rewriting stage rather than simple function-name extraction. Compared with LLMLingua-2 at matched compression rates, our method consistently achieves superior OckScore performance across all evaluated backends. These results demonstrate that proactive prompt optimization can substantially reduce inference costs without sacrificing coding quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。