让大模型生成代码时能自我修正，提升安全性。
Autoregressive, Yet Revisable: In Decoding Revision for Secure Code Generation
- 用特殊指令令牌实现模型在生成中即时回溯修改。
- 在安全代码生成任务中漏洞率显著降低，推理开销极小。
- 适合需要高可靠性代码的开发场景，如金融、医疗系统。
基于大语言模型的代码生成通常被建模为严格单调的过程，即线性地向不可变前缀添加标记。这一方式与编程中前后生成与实时修订交织的认知过程相悖。以往工作尝试通过后置代理或外部静态工具引入修订，但要么延迟高，要么无法利用模型内在语义推理能力。本文提出「修订流」（Stream of Revision），将代码生成从单调流程转变为动态自修正轨迹，充分利用模型自身能力。我们引入特定动作标记，使模型能在单次前向传播中无缝回溯并编辑自身历史。通过内化修订循环，该框架使模型能够就地激活潜在能力，无需外部依赖。在安全代码生成任务上的实证结果表明，该方法显著降低漏洞数量，且推理开销极低。
原文摘要 · Abstract (English)
Large Language Model (LLM) based code generation is predominantly formulated as a strictly monotonic process, appending tokens linearly to an immutable prefix. This formulation contrasts to the cognitive process of programming, which is inherently interleaved with forward generation and on-the-fly revision. While prior works attempt to introduce revision via post-hoc agents or external static tools, they either suffer from high latency or fail to leverage the model's intrinsic semantic reasoning. In this paper, we propose Stream of Revision, a paradigm shift that elevates code generation from a monotonic stream to a dynamic, self-correcting trajectory by leveraging model's intrinsic capabilities. We introduce specific action tokens that enable the model to seamlessly backtrack and edit its own history within a single forward pass. By internalizing the revision loop, our framework Stream of Revision allows the model to activate its latent capabilities just-in-time without external dependencies. Empirical results on secure code generation show that Stream of Revision significantly reduces vulnerabilities with minimal inference overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。