无意提示词能决定大模型代码生成的算法选择，影响性能与安全。
The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation
- 通过4.6万次实验验证提示词细节可系统性引导算法选择
- 最高达100个百分点的算法分布变化，与提示语义强相关
- 直接命名算法最有效，适合关注代码质量的开发者
大型语言模型（LLMs）现广泛生成生产级代码，许多任务存在多个合法算法解。偶然的提示词线索（如上下文词汇或元数据）会引导模型选择特定算法，即使所有输出均通过相同测试。尽管提示敏感性被用于提升输出质量，但本文关注在正确性固定的条件下，算法选择的输出策略。我们定义算法引导为提示引起的算法族分布偏移，进行了46,535次受控实验，覆盖11个任务、19种提示类型（含18个通道及一项保留语义但改变排版和标点的缓存语义-表面对比），以及15种模型配置。结果发现算法族分布出现显著且系统性的偏移（最高达100个百分点），且主要与提示语义一致，甚至在实际任务如限流中也观察到此现象。直接命名算法是测试中最有效的缓解方式。因此，意外上下文造成了对性能、安全性和可维护性的“隐形彩票”。
原文摘要 · Abstract (English)
Large language models (LLMs) now generate substantial production code, often for tasks with multiple valid algorithmic solutions. Incidental prompt cues, meaning contextual words or metadata outside the task specification, can steer which algorithm the model selects, even when all outputs pass the same tests. Prompt sensitivity is well studied as a tool to improve output quality. Here, output policy means algorithm choice under fixed correctness. We define algorithm steering as cue-induced shifts in algorithm-family distributions and run 46,535 controlled experiments across 11 tasks, 19 cue types (18 channels plus a memoization semantic-vs-surface ablation that preserves meaning while changing typography and punctuation), and 15 model configurations. We find large, systematic shifts in algorithm-family distributions (up to 100 pp), largely consistent with cue semantics, including in applied tasks such as rate limiting. Direct algorithm naming is the most reliable mitigation we tested. Accidental context therefore creates an "invisible lottery" over performance, security, and maintainability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。