用领域预训练提升域名窃取检测，降低误报率。
Improving DNS Exfiltration Detection via Transformer Pretraining
- 用领域数据预训练BERT，再微调检测子域名窃取
- 在低误报率下，检测性能显著优于随机初始化模型
- 预训练步数越多，标注数据越充足时效果越明显
我们研究了在领域内对双向编码器表示的Transformer（BERT）模型进行预训练，是否能提升在低误报率条件下对子域名级数据外泄行为的检测能力。以往工作多聚焦于微调通用Transformer模型，未系统分离预训练对下游分类任务的影响。为此，我们构建了受控实验流程，在验证集上冻结操作点并转移到测试集，实现不同标签和预训练预算下的清晰对比。结果表明，在接收者操作特征曲线（ROC）的左尾区域有显著改进，尤其优于随机初始化基线。此外，在预训练模型中，当微调可用标注数据较多时，增加预训练步数带来的收益最大。
原文摘要 · Abstract (English)
We study whether in-domain pretraining of Bidirectional Encoder Representations from Transformer (BERT) model improves subdomain-level detection of exfiltration at low false positive rates. While previous work mostly examines fine-tuned generic Transformers, it does not aim to isolate the effect of pretraining on the downstream task of classification. To address this gap, we develop a controlled pipeline where we freeze operating points on validation and transfer them to the test set, thus enabling clean ablations across different label and pretraining budgets. Our results show significant improvements in the left tail of the Receiver Operating Characteristic (ROC) curve, especially against randomly initialized baseline. Additionally, within pretrained model variants, increasing the number of pretraining steps helps the most when more labeled data are available for fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。