用稀疏自编码器从预训练模型中提取特征，无需微调即可高效检测Java漏洞
Are Sparse Autoencoders Useful for Java Function Bug Detection?
- 利用预训练模型内部表示，通过稀疏自编码器提取可解释特征
- 在不微调模型条件下实现最高89%的F1分数，优于微调基线
- 适合关注模型可解释性与轻量级安全检测的研究者和开发者
软件漏洞如缓冲区溢出和SQL注入是安全事件的主要根源。传统检测方法虽重要，但受限于高误报率、扩展性差和人工依赖。这推动了基于AI的自动化漏洞检测与安全代码生成研究。尽管大语言模型（LLMs）为分类任务带来新可能，其复杂性和不可解释性阻碍了可解释性与部署。稀疏自编码器（SAEs）为此提供了一种有前景的解决方案。本文探究SAE是否可作为轻量、可解释的Java函数漏洞检测替代方案。我们评估了SAE在GPT-2 Small和Gemma 2B表示上的效果，考察其在不微调底层LLM的前提下，能否有效识别缺陷行为。结果表明，基于SAE的特征使漏洞检测达到最高89%的F1分数，持续优于微调的Transformer编码器基线。本工作首次提供了实证证据：SAE可直接从预训练模型内部表示中检测软件漏洞，无需微调或任务特定监督。代码已开源：https://github.com/rufimelo99/SAE-Java-Bug-Detection
原文摘要 · Abstract (English)
Software vulnerabilities such as buffer overflows and SQL injections are a major source of security breaches. Traditional methods for vulnerability detection remain essential but are limited by high false positive rates, scalability issues, and reliance on manual effort. These constraints have driven interest in AI-based approaches to automated vulnerability detection and secure code generation. While Large Language Models (LLMs) have opened new avenues for classification tasks, their complexity and opacity pose challenges for interpretability and deployment. Sparse Autoencoder offer a promising solution to this problem. We explore whether SAEs can serve as a lightweight, interpretable alternative for bug detection in Java functions. We evaluate the effectiveness of SAEs when applied to representations from GPT-2 Small and Gemma 2B, examining their capacity to highlight buggy behaviour without fine-tuning the underlying LLMs. We found that SAE-derived features enable bug detection with an F1 score of up to 89%, consistently outperforming fine-tuned transformer encoder baselines. Our work provides the first empirical evidence that SAEs can be used to detect software bugs directly from the internal representations of pretrained LLMs, without any fine-tuning or task-specific supervision. Code available at https://github.com/rufimelo99/SAE-Java-Bug-Detection
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。