跨数据集的钓鱼网址特征可信吗?研究发现特征依赖数据集,不能通用。
Can Features for Phishing URL Detection Be Trusted Across Diverse Datasets? A Case Study with Explainable AI
- 对比两个公开数据集,分析特征在不同场景下的表现差异。
- 相同特征在不同数据集中贡献度差异大,模型跨数据集泛化能力差。
- 使用SHAP等可解释AI方法揭示特征可靠性问题,适合安全研究人员参考。
钓鱼攻击通过伪装成可信实体诱骗用户泄露敏感信息,已成为普遍的网络威胁。近年来,基于提取的网址字符串和网站内容特征,利用监督式机器学习模型实现钓鱼网址检测已被广泛采纳。然而,这些特征是否具有普适性仍不明确。本文以两个公开钓鱼网址数据集为案例,分析其各自独有的及重叠的特征。通过实证实验与可解释人工智能(XAI)方法(如SHAP图),探究重叠特征在不同数据集中的相似性,以及模型在一种数据集训练后在另一数据集上的表现。结果表明,尽管特征行为类似,但其实际贡献存在显著差异,说明钓鱼网址检测特征往往依赖特定数据集,难以跨数据集通用。
原文摘要 · Abstract (English)
Phishing has been a prevalent cyber threat that manipulates users into revealing sensitive private information through deceptive tactics, designed to masquerade as trustworthy entities. Over the years, proactively detection of phishing URLs (or websites) has been established as an widely-accepted defense approach. In literature, we often find supervised Machine Learning (ML) models with highly competitive performance for detecting phishing websites based on the extracted features from both phishing and benign (i.e., legitimate) websites. However, it is still unclear if these features or indicators are dependent on a particular dataset or they are generalized for overall phishing detection. In this paper, we delve deeper into this issue by analyzing two publicly available phishing URL datasets, where each dataset has its own set of unique and overlapping features related to URL string and website contents. We want to investigate if overlapping features are similar in nature across datasets and how does the model perform when trained on one dataset and tested on the other. We conduct practical experiments and leverage explainable AI (XAI) methods such as SHAP plots to provide insights into different features' contributions in case of phishing detection to answer our primary question, "Can features for phishing URL detection be trusted across diverse dataset?". Our case study experiment results show that features for phishing URL detection can often be dataset-dependent and thus may not be trusted across different datasets even though they share same set of feature behaviors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。