arXiv:2512.15003cs.CRcs.LG2025-12中稿 · publication at SAN…

提出SEBERTIS框架,让模型不依赖关键词也能精准识别安全漏洞报告。

SeBERTis: A Framework for Producing Classifiers of Security-Related Issue Reports

  • 用语义替代词微调BERT,避免模型只记关键词做判断。
  • 在1万条GitHub报告上达到0.9880的F1分数,显著优于现有方法。
  • 适合需要高精度实时检测安全问题的软件维护团队使用。

监控问题追踪器提交是软件维护的关键任务,核心目标是优先处理高风险的安全相关缺陷。若能早期识别此类缺陷,可降低其传播至依赖产品及损害利益相关者的风险。为辅助分类工程师,已有多种自动检测技术被提出,涵盖机器学习模型与提示大语言模型(LLMs)。然而,现有方法常依赖词汇线索作为决策捷径,对复杂报告检测率低,尚未满足实时检测的实际需求。为此,本文提出SEBERTIS框架,通过微调双向变压器架构作为掩码语言模型(MLM),在一系列语义等价的预测标签(称为语义替代词)被替换为掩码时进行训练,使深度神经网络分类器摆脱对词汇线索的依赖,从而自信识别完全未见过的安全相关问题。SEBERTIS训练的分类器在10,000条精选GitHub问题报告数据集上取得0.9880的F1分数,显著优于当前最先进方法,检测精度、召回率和F1分数分别比基于ML的基线高出14.44%-96.98%、15.40%-93.07%、14.90%-94.72%;同时大幅超越LLM基线,提升幅度分别为23.20%-63.71%、36.68%-85.63%、39.49%-74.53%。

原文摘要 · Abstract (English)

Monitoring issue tracker submissions is a crucial software maintenance activity. A key goal is the prioritization of high risk, security-related bugs. If such bugs can be recognized early, the risk of propagation to dependent products and endangerment of stakeholder benefits can be mitigated. To assist triage engineers with this task, several automatic detection techniques, from Machine Learning (ML) models to prompting Large Language Models (LLMs), have been proposed. Although promising to some extent, prior techniques often memorize lexical cues as decision shortcuts, yielding low detection rate specifically for more complex submissions. As such, these classifiers do not yet reach the practical expectations of a real-time detector of security-related issues. To address these limitations, we propose SEBERTIS, a framework to train Deep Neural Networks (DNNs) as classifiers independent of lexical cues, so that they can confidently detect fully unseen security-related issues. SEBERTIS capitalizes on fine-tuning bidirectional transformer architectures as Masked Language Models (MLMs) on a series of semantically equivalent vocabulary to prediction labels (which we call Semantic Surrogates) when they have been replaced with a mask. Our SEBERTIS-trained classifier achieves a 0.9880 F1-score in detecting security-related issues of a curated corpus of 10,000 GitHub issue reports, substantially outperforming state-of-the-art issue classifiers, with 14.44%-96.98%, 15.40%-93.07%, and 14.90%-94.72% higher detection precision, recall, and F1-score over ML-based baselines. Our classifier also substantially surpasses LLM baselines, with an improvement of 23.20%-63.71%, 36.68%-85.63%, and 39.49%-74.53% for precision, recall, and F1-score.

安全检测自然语言处理深度学习代码质量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。