arXiv:2502.20246cs.CL2025-02被引 2

提出新方法检测代码数据中的无效代码污染,提升安全性和准确性。

Beyond Natural Language Perplexity: Detecting Dead Code Poisoning in Code Generation Datasets

  • 基于代码行上下文关系计算逐行困惑度,识别异常代码行
  • 检测F1分数提升0.14-0.19,定位精度提高44%-65%
  • 速度提升0.62-23倍,适合大规模代码数据清洗

大型语言模型在代码任务中的广泛应用引发了训练数据安全性的担忧。其中,死代码污染是一种关键威胁:将语法正确但功能冗余的代码注入训练数据,可操纵模型行为,导致神经代码搜索系统性能下降,产生偏见或不安全的代码建议。现有方法如令牌级困惑度分析,因编程语言的结构和上下文特性难以有效识别死代码。本文提出DePA(Dead Code Perplexity Analysis),一种针对代码结构特性的行级检测与清洗方法。DePA通过利用代码行间的上下文关系计算行级困惑度,并通过比较行困惑度与文件内整体分布来识别异常行。在基准数据集上的实验表明,DePA显著优于现有方法,检测F1分数提升0.14–0.19,中毒片段定位精度提高44%–65%。此外,检测速度提升0.62–23倍,适用于大规模数据清洗。总体而言,DePA为保障代码生成模型训练数据完整性提供了高效可靠的解决方案。

原文摘要 · Abstract (English)

The increasing adoption of large language models (LLMs) for code-related tasks has raised concerns about the security of their training datasets. One critical threat is dead code poisoning, where syntactically valid but functionally redundant code is injected into training data to manipulate model behavior. Such attacks can degrade the performance of neural code search systems, leading to biased or insecure code suggestions. Existing detection methods, such as token-level perplexity analysis, fail to effectively identify dead code due to the structural and contextual characteristics of programming languages. In this paper, we propose DePA (Dead Code Perplexity Analysis), a novel line-level detection and cleansing method tailored to the structural properties of code. DePA computes line-level perplexity by leveraging the contextual relationships between code lines and identifies anomalous lines by comparing their perplexity to the overall distribution within the file. Our experiments on benchmark datasets demonstrate that DePA significantly outperforms existing methods, achieving 0.14-0.19 improvement in detection F1-score and a 44-65% increase in poisoned segment localization precision. Furthermore, DePA enhances detection speed by 0.62-23x, making it practical for large-scale dataset cleansing. Overall, by addressing the unique challenges of dead code poisoning, DePA provides a robust and efficient solution for safeguarding the integrity of code generation model training datasets.

代码安全数据清洗大模型检测方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。