用智能重播提升代码漏洞检测模型抗遗忘能力。
Enhancing Continual Learning for Software Vulnerability Prediction: Addressing Catastrophic Forgetting via Hybrid-Confidence-Aware Selective Replay for Temporal LLM Fine-Tuning
- 基于置信度选择不确定样本重播，保持漏洞与修复样本平衡
- 在时间序列测试中宏F1达0.667，比基线提升0.016（p=0.026）
- 适合需要持续更新的漏洞检测系统，兼顾准确率与训练效率
近期研究将大语言模型（LLMs）应用于源码漏洞检测，但多数评估仍依赖随机划分训练测试集，忽略时间维度，高估实际表现。现实中，检测器需在不断演化的代码库上识别未来漏洞，面临时间分布漂移问题。本文对微软Phi-2模型（采用LoRA微调）在2018–2024年CVE关联数据集上进行持续学习微调，按双月划分时间窗口。评估八种持续学习策略，包括仅窗口训练、累计训练及基于重播和正则化的基线方法。提出混合类感知选择性重播（Hybrid-CASR），一种针对二分类漏洞检测的置信度感知重播方法，优先保留不确定样本并维持漏洞/修复函数的平衡比例。在双月前向评估中，Hybrid-CASR取得0.667的宏F1，优于窗口仅训练基线（0.651）0.016，统计显著（p=0.026），且后向保留能力更强（IBR@1为0.741）。该方法每窗口训练时间减少约17%，而累计训练仅提升F1至0.661，但计算成本高达15.9倍。结果表明，结合类别平衡的选择性重播在持续时间漂移下提供了实用的精度-效率权衡。
原文摘要 · Abstract (English)
Recent work applies Large Language Models (LLMs) to source-code vulnerability detection, but most evaluations still rely on random train-test splits that ignore time and overestimate real-world performance. In practice, detectors are deployed on evolving code bases and must recognise future vulnerabilities under temporal distribution shift. This paper investigates continual fine-tuning of a decoder-style language model (microsoft/phi-2 with LoRA) on a CVE-linked dataset spanning 2018-2024, organised into bi-monthly windows. We evaluate eight continual learning strategies, including window-only and cumulative training, replay-based baselines and regularisation-based variants. We propose Hybrid Class-Aware Selective Replay (Hybrid-CASR), a confidence-aware replay method for binary vulnerability classification that prioritises uncertain samples while maintaining a balanced ratio of VULNERABLE and FIXED functions in the replay buffer. On bi-monthly forward evaluation Hybrid-CASR achieves a Macro-F1 of 0.667, improving on the window-only baseline (0.651) by 0.016 with statistically significant gains ($p = 0.026$) and stronger backward retention (IBR@1 of 0.741). Hybrid-CASR also reduces training time per window by about 17 percent compared to the baseline, whereas cumulative training delivers only a minor F1 increase (0.661) at a 15.9-fold computational cost. Overall, the results show that selective replay with class balancing offers a practical accuracy-efficiency trade-off for LLM-based temporal vulnerability detection under continuous temporal drift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。