通过挖掘模型内部表示,实现代码生成时的精准安全引导。
Security-by-Design for LLM-Based Code Generation: Leveraging Internal Representations for Concept-Driven Steering Mechanisms
- 分析模型内部安全概念表征,发现其能识别漏洞子概念。
- 在多个基准上优于现有方法,显著提升生成代码安全性。
- 轻量模块化设计,可无缝集成到现有代码生成模型中。
大型语言模型(LLMs)在理解自然语言和生成复杂代码方面表现出色,但随着开发者将代码生成模型用于关键任务,研究发现这些模型常生成功能正确却存在安全隐患的代码,带来重大风险。尽管已有多种方法试图提升生成代码的安全性,但综合基准测试显示,现有方法在功能正确性和安全性上仅取得有限改进,难以满足实际需求。这源于对代码生成内部机制及漏洞根本原因的理解不足,导致研究依赖启发式方法和经验观察。本文深入探究代码生成模型中安全概念的内部表征,发现模型在生成不安全代码时,已具备对漏洞的认知。系统评估表明,代码生成模型能够区分不同安全子概念,实现比以往黑盒方法更精细的分析。基于此,我们提出面向代码生成的「安全概念引导」机制(SCS-Code),在生成过程中动态引导模型内部表示向安全且功能正确的方向演化,形成一种轻量、模块化的可集成方案。实验表明,该方法在多个安全编码基准上优于当前最优方法。
原文摘要 · Abstract (English)
Large Language Models (LLMs) show remarkable capabilities in understanding natural language and generating complex code. However, as practitioners adopt CodeLLMs for increasingly critical development tasks, research reveals that these models frequently generate functionally correct yet insecure code, posing significant security risks. While multiple approaches have been proposed to improve security in AI-based code generation, combined benchmarks show these methods remain insufficient for practical use, achieving only limited improvements in both functional correctness and security. This stems from a fundamental gap in understanding the internal mechanisms of code generation and the root causes of security vulnerabilities, forcing researchers to rely on heuristics and empirical observations. In this work, we investigate the internal representation of security concepts in CodeLLMs, revealing that models are often aware of vulnerabilities as they generate insecure code. Through systematic evaluation, we demonstrate that CodeLLMs can distinguish between security subconcepts, enabling a more fine-grained analysis than prior black-box approaches. Leveraging these insights, we propose Secure Concept Steering for CodeLLMs (SCS-Code). During token generation, SCS-Code steers LLMs' internal representations toward secure and functional code output, enabling a lightweight and modular mechanism that can be integrated into existing code models. Our approach achieves superior performance compared to state-of-the-art methods across multiple secure coding benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。