系统评估对抗训练在恶意软件检测中的效果,发现真实攻击下收益有限。
On the Effectiveness of Adversarial Training on Malware Classifiers
- 构建多维度评估框架Rubik,整合数据、特征、模型等变量
- 实证显示真实对抗样本仅在特定条件下提升鲁棒性
- 揭示模型结构和特征空间对防御效果的关键影响
对抗训练(AT)是应对机器学习逃避攻击的重要防御手段,但在真实恶意软件检测场景中的有效性仍不明确。这一不确定性源于前期研究的严重断层:多数工作忽视了恶意软件的本质特性,且研究碎片化,孤立考察对抗样本的真实性或置信度等变量,或依赖弱评估方式,导致结论不可推广。为此,我们提出Rubik框架,实现恶意软件领域中对抗训练的系统性、多维评估。该框架定义了数据、特征表示、分类器及鲁棒优化设置等关键维度,通过真实逃避攻击等可靠评估实践,全面探索各变量间的相互作用。我们在Android恶意软件上实例化Rubik,实证分析这些因素如何共同塑造模型鲁棒性。研究结果挑战既有认知——例如,真实可实现的对抗样本仅带来条件性鲁棒性提升——并揭示新洞见,如模型架构与特征空间结构在决定AT成败中的核心作用。基于此,我们提炼出四项关键洞察,揭露四项常见评估误区,并提出实用建议,以指导真正鲁棒的恶意软件分类器开发。
原文摘要 · Abstract (English)
Adversarial Training (AT) is a key defense against Machine Learning evasion attacks, but its effectiveness for real-world malware detection remains poorly understood. This uncertainty stems from a critical disconnect in prior research: studies often overlook the inherent nature of malware and are fragmented, examining diverse variables like realism or confidence of adversarial examples in isolation, or relying on weak evaluations that yield non-generalizable insights. To address this, we introduce Rubik, a framework for the systematic, multi-dimensional evaluation of AT in the malware domain. This framework defines diverse key factors across essential dimensions, including data, feature representations, classifiers, and robust optimization settings, for a comprehensive exploration of the interplay of influential AT's variables through reliable evaluation practices, such as realistic evasion attacks. We instantiate Rubik on Android malware, empirically analyzing how this interplay shapes robustness. Our findings challenge prior beliefs--showing, for instance, that realizable adversarial examples offer only conditional robustness benefits--and reveal new insights, such as the critical role of model architecture and feature-space structure in determining AT's success. From this analysis, we distill four key insights, expose four common evaluation misconceptions, and offer practical recommendations to guide the development of truly robust malware classifiers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。