对比多种学习型相似性技术,发现无单一方法最优。
A Unified Evaluation of Learning-Based Similarity Techniques for Malware Detection
- 统一框架下评估多种学习型相似性方法
- 不同方法在准确率与效率上各有优劣
- 适合需要多策略的恶意软件分析场景
加密摘要(如 MD5、SHA-256)对输入微小变化极为敏感,难以用于威胁狩猎、恶意软件分析等实际任务。相似性技术通过近似匹配,使相关字节序列生成可度量相似的指纹。现代企业面临数十万终端、数十亿文件,技术的有效性与可扩展性至关重要。已有方法包括相似性摘要(如 ssdeep、sdhash、TLSH)和基于机器学习的嵌入生成方法,但多在孤立环境下评估,使用不同数据集与指标。本文首次在统一实验框架下,使用公开大样本数据集系统比较各类学习型相似性技术,采用行业标准指标进行评估。结果表明,无单一方法在所有维度表现最佳,各方法存在显著权衡,提示安全平台应结合互补的分类与相似性技术,而非依赖单一方法。
原文摘要 · Abstract (English)
Cryptographic digests (e.g., MD5, SHA-256) are designed to provide exact identity. Any single-bit change in the input produces a completely different hash, which is ideal for integrity verification but limits their usefulness in many real-world tasks like threat hunting, malware analysis and digital forensics, where adversaries routinely introduce minor transformations. Similarity-based techniques address this limitation by enabling approximate matching, allowing related byte sequences to produce measurably similar fingerprints. Modern enterprises manage tens of thousands of endpoints with billions of files, making the effectiveness and scalability of the proposed techniques more important than ever in security applications. Security researchers have proposed a range of approaches, including similarity digests and locality-sensitive hashes (e.g., ssdeep, sdhash, TLSH), as well as more recent machine-learning-based methods that generate embeddings from file features. However, these techniques have largely been evaluated in isolation, using disparate datasets and evaluation criteria. This paper presents a systematic comparison of learning-based classification and similarity methods using large, publicly available datasets. We evaluate each method under a unified experimental framework with industry-accepted metrics. To our knowledge, this is the first reproducible study to benchmark these diverse learning-based similarity techniques side by side for real-world security workloads. Our results show that no single approach performs well across all dimensions; instead, each exhibits distinct trade-offs, indicating that effective malware analysis and threat-hunting platforms must combine complementary classification and similarity techniques rather than rely on a single method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。