arXiv:2604.14210cs.CLcs.SE2026-04被引 2

中文提示未必更省 token,反而成功率更低,别盲目换语言省钱。

Chinese Language Is Not More Efficient Than English in Vibe Coding: A Preliminary Study on Token Cost and Problem-Solving Rate

  • 用 SWE-bench Lite 测了中英文提示的编码效率
  • 中文平均耗 token 更多,且任务成功率达不到英文水平
  • 不同模型表现差异大,不能一概而论

社交媒体和开发者社区流传一种说法:中文提示在大模型编程任务中比英文更高效,可能节省高达40%的API成本。本文基于SWE-bench Lite基准,对多个模型进行了实证研究。结果发现:第一,中文并无明显的效率优势;第二,不同模型架构下中文提示的token消耗不一致——MiniMax-2.7中中文耗1.28倍更多token,而GLM-5却更少;第三,所有模型在中文提示下的任务成功率普遍低于英文。进一步计算每成功任务的预期成本(综合考虑token消耗与解决率)后,仍无成本优势。结论为初步证据,受限于模型与数据集数量,但已表明语言影响是模型依赖的,不可仅因换语言就期待降本增效。

原文摘要 · Abstract (English)

A claim has been circulating on social media and practitioner forums that Chinese prompts are more token-efficient than English for LLM coding tasks, potentially reducing costs by up to 40\%. This claim has influenced developers to consider switching to Chinese for ``vibe coding'' to save on API costs. In this paper, we conduct a rigorous empirical study using SWE-bench Lite, a benchmark of software engineering tasks, to evaluate whether this claim of Chinese token efficiency holds up to scrutiny. Our results reveal three key findings: First, the efficiency advantage of Chinese is not observed. Second, token cost varies by model architecture in ways that defy simple assumptions: while MiniMax-2.7 shows 1.28x higher token costs for Chinese, GLM-5 actually consumes fewer tokens with Chinese prompts. Third, and most importantly, we found that the success rate when prompting in Chinese is generally lower than in English across all models we tested. We also measure cost efficiency as expected cost per successful task -- jointly accounting for token consumption and task resolution rate. These findings should be interpreted as preliminary evidence rather than a definitive conclusion, given the limited number of models evaluated and the narrow set of benchmarks tested due to resource constraints; they indicate that language effects on token cost are model-dependent, and that practitioners should not expect cost savings or performance gains just by switching their prompt language to Chinese.

提示工程语言效率成本优化LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。