用分层惩罚信号提升Python漏洞预测，强化学习效果最佳。
From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python
- 用归一化惩罚构造密集奖励，通过强化学习优化模型
- 最优策略使Qwen2.5-Coder-7B在SVEN数据集上惩罚值下降27.9%
- 适合关注代码安全与模型优化的研究者
原始ALPHA基准引入了面向分类的惩罚机制来评估Python中的CWE级漏洞预测，并提出该惩罚理论上也可作为训练信号。本文验证了这一设想。我们比较了三种传递方式：监督微调、双头分类损失和基于归一化惩罚的密集奖励强化学习（GRPO）。结果表明，监督方法在分布外情况下持续低于零样本基线，而GRPO表现优异。最优策略在贪婪解码下将Qwen2.5-Coder-7B在Security Hardening和Adversarial Testing（SVEN）数据集上的累计ALPHA惩罚降低27.9%，采样解码下降低25.5%（p = 0.005，Welch's t检验），达到与4.5倍大的零样本教师模型统计等效水平。结论指出，分层惩罚作为训练信号的有效性高度依赖其传递方式的直接性。
原文摘要 · Abstract (English)
The original ALPHA benchmark introduced a taxonomy-aware penalty for evaluating CWE-level vulnerability prediction in Python and proposed that the penalty could theoretically also serve as a training signal. This paper provides that validation. We compare three delivery mechanisms: supervised fine-tuning, a dual-head classification loss, and reinforcement learning with a dense reward derived from the normalised penalty. We find that supervised approaches consistently regress below the zero-shot baseline under distribution shift, while GRPO succeeds. Our best policy reduces the cumulative ALPHA penalty of Qwen2.5-Coder-7B on Security Hardening and Adversarial Testing (SVEN) dataset by 27.9% under greedy decoding, and by 25.5% under sampled decoding(p = 0.005, Welch's t-test), reaching statistical parity with its 4.5x larger zero-shot teacher. We conclude that the value of a hierarchical penalty as a training signal depends largely on the directness of its delivery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。