构建基准框架，评估代码大模型在安卓恶意软件分析中的表现
On Benchmarking Code LLMs for Android Malware Analysis
- 设计结构化输出规范，支持恶意函数识别与恶意目的总结
- 基于118个样本、超750万函数的基准数据集，量化模型性能
- 揭示模型对函数重命名的敏感性，适合安全研究者参考
大型语言模型在代码智能任务中表现出色，但在安卓恶意软件分析方面的效果仍不明确。反编译后的安卓恶意软件代码因恶意逻辑隐藏在大量函数中且函数名无意义，带来独特挑战。本文提出CAMA框架，系统评估代码大模型在安卓恶意软件分析中的能力。CAMA定义了结构化模型输出，支持恶意函数识别和恶意目的总结，并集成一致性、保真度和语义相关性三个领域特定评估指标，实现模型稳定性和有效性评估及跨模型比较。我们构建了一个包含118个近期样本、来自13个家族的基准数据集，涵盖超过750万条唯一函数，使用CAMA评估了四种主流开源代码大模型。实验揭示了代码大模型对反编译代码的理解机制，并量化了其对函数重命名的敏感性，凸显其潜力与当前局限。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated strong capabilities in various code intelligence tasks. However, their effectiveness for Android malware analysis remains underexplored. Decompiled Android malware code presents unique challenges for analysis, due to the malicious logic being buried within a large number of functions and the frequent lack of meaningful function names. This paper presents CAMA, a benchmarking framework designed to systematically evaluate the effectiveness of Code LLMs in Android malware analysis. CAMA specifies structured model outputs to support key malware analysis tasks, including malicious function identification and malware purpose summarization. Built on these, it integrates three domain-specific evaluation metrics (consistency, fidelity, and semantic relevance), enabling rigorous stability and effectiveness assessment and cross-model comparison. We construct a benchmark dataset of 118 Android malware samples from 13 families collected in recent years, encompassing over 7.5 million distinct functions, and use CAMA to evaluate four popular open-source Code LLMs. Our experiments provide insights into how Code LLMs interpret decompiled code and quantify the sensitivity to function renaming, highlighting both their potential and current limitations in malware analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。