拆解安全分类中半监督学习与分类器调优的贡献,发现调参分类器即可接近最优效果。
SemiScope: Disentangling Classifier Tuning and Joint Optimization in Semi-Supervised Security Classification

- 用贝叶斯优化联合调优半监督设置、置信度过滤、过采样和分类器
- 在10%标签下,新方法比默认基线提升0.7-12.7个点,四组数据等效于完整调优
- 仅调分类器+阈值就能收回86%收益,适合资源有限的场景
安全分类标注数据稀缺。半监督学习(SSL)可将少量标注数据推广至大量未标注数据。然而,安全应用常将SSL当作黑箱:使用默认参数、固定分类器且忽略伪标签引发的类别不平衡。本文通过构建SemiScope分析工具,分离了分类器调优与联合优化的影响。该工具采用贝叶斯优化联合调优SSL配置、置信度过滤、过采样及分类器。控制实验Tuned-Clf固定SSL为默认,但享有相同的100次分类器超参数搜索预算和验证集阈值调优。在10%标签率下,采用配对TOST检验(最小显著差异±1.0的g-measure),结果表明:SemiScope优于所有默认基线,在五个数据集上提升0.7-12.7点;在同等预算下,Tuned-Clf在四组数据上与完整管道无统计差异,仅Phishing不显著。仅分类器调优即恢复了半监督收益的86%(相对于默认自训练+随机森林)。结论:关键贡献在于分解协议。更简单的方案已足够:使用自训练,用贝叶斯优化调分类器,并在验证集上调阈值。该方法在四个数据集上达到与监督式随机森林相近性能(20-30%标签),在Drebin上为40%标签,且所需标签率低于默认自训练+随机森林。
原文摘要 · Abstract (English)
Background. Labeled data for security classification is scarce. Semi-supervised learning (SSL) propagates labels from a small labeled pool to larger unlabeled pools. Yet security applications often use SSL as a black box: default parameters, a fixed classifier, and no handling of pseudo-label-induced class imbalance. Aims. Recent work reports sizeable gains from optimizing SSL pipelines via joint search, AutoML, or per-component tuning. These gains are hard to attribute: they may reflect useful SSL-classifier interactions, or mostly from simply tuning the downstream classifier. We disentangle these effects for binary tabular security data with classical SSL and tree-based classifiers. Method. We build SemiScope as an analysis instrument, not a deployment recommendation. It uses Bayesian Optimization to jointly tune SSL settings, confidence filtering, oversampling, and the classifier. The key control, Tuned-Clf, fixes SSL to defaults but gets the same 100-trial classifier budget and validation-set threshold tuning as SemiScope. At 10% labels, we compare them with paired TOST using a +/-1.0 g-measure smallest effect of interest. Results. SemiScope beats every default SSL baseline on all five datasets, improving over the strongest by 0.7-12.7 points. Under the equal-budget control, Tuned-Clf is statistically equivalent to the full pipeline on 4 of 5 datasets; Phishing is inconclusive. Classifier HPO alone recovers a median 86% of SemiScope's gain over Default Self-Training (ST) + Random Forest (RF). Conclusions. The reusable contribution is the decomposition protocol. A simpler recipe suffices: use Self-Training, tune the classifier with Bayesian Optimization, and tune the decision threshold on validation data. It reaches within 1 g-measure of Supervised RF at 20-30% labels on four datasets and 40% on Drebin, at the same or lower label rate than Default ST + RF on every dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。