让大模型生成代码时更懂用户意图，提升准确率。
Pragmatic Reasoning improves LLM Code Generation
- 通过局部对比推理，从候选代码中选出最符合原指令的
- 在多个基准上平均准确率超越现有方法，12组中有10组最优
- 适合需要精准理解指令的代码生成场景
语用推理使对话者基于共享背景和反事实可能，推断出模糊或不完整信息的真正意图。自然语言转代码生成也面临类似挑战，用户指令常对应多个合理程序。但直接采用RSA风格推理困难，因需对大规模程序与替代指令空间进行概率估计。我们提出CodeRSA，一种受RSA启发的重排序方法，通过在采样代码候选间进行局部语用竞争，使语用推理可计算。CodeRSA构建候选代码诱导的替代指令，并估算哪些候选最显著地被原始指令支持，避免对全程序-指令空间进行全局归一化。我们在HumanEval+、MBPP+和BigCodeBench上使用四个开源指令遵循模型评估CodeRSA，结果表明其在12个模型-基准组合中，有10个达到最高平均准确率，其余亦具竞争力。进一步分析显示，其优势来自局部成对语用比较与全局支持的结合，为自然语言不确定性下的语言到代码重排序提供了可扩展方向。
原文摘要 · Abstract (English)
Pragmatic reasoning helps interlocutors infer intended meaning from ambiguous or underspecified messages by considering shared context and counterfactual alternatives. Similar challenges arise in natural language-to-code generation, where user instructions often admit multiple plausible candidate programs. However, direct RSA-style inference is difficult because it requires probability estimation over large spaces of programs and alternative instructions. We propose CodeRSA, an RSA-motivated reranking method that makes pragmatic reasoning tractable through local pragmatic contests among sampled code candidates. CodeRSA constructs candidate-induced alternative instructions and estimates which candidates are most distinctively supported by the original instruction, avoiding global normalization over the full program-instruction space. We evaluate CodeRSA on HumanEval+, MBPP+, and BigCodeBench using four open-weight instruction-following models. CodeRSA achieves the strongest average accuracy in 10 of 12 model-benchmark settings and remains competitive in the remaining cases. Further analyses show that its gains come from combining local pairwise pragmatic comparison with broader global support, suggesting a scalable direction for language-to-code reranking under natural-language uncertainty.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。