让大模型直接理解二进制代码，提升恶意软件分析效率。
Large Byte Model: Teaching Language Models About Compiled Code
- 用自定义字节分词器构建原生处理二进制的LLM
- 恶意软件家族分类准确率达69%，架构识别达98%
- 需加入领域知识训练，通用模型效果差
恶意软件分析始于可执行程序的原始字节，现有工具将这些字节转换为汇编等高层表示成本高且易出错。大型语言模型（LLMs）无法直接处理原始字节并回答相关问题。为此，我们提出首个原生处理字节的LLM。基于一种定制的字节分词技术，该模型能对恶意软件二进制文件回答复杂问题，准确率在恶意软件家族分类中达69%，架构分类高达98%。研究发现，在训练中注入领域知识对本应用至关重要——通用模型既缺乏准确性也无洞察力。我们已将此方案部署给少量分析师以收集反馈，用于后续优化。
原文摘要 · Abstract (English)
Malware analysis starts with the raw bytes of an executable program, and tools to "lift" these to higher-level representations, such as assembly, are expensive and subject to error. Large Language Models (LLMs) cannot process raw byte representations and answer questions about them. To this end, we present the first byte-native LLM. Based on a vocabulary expansion technique using a bespoke byte tokenizer, such a model is capable of responding to complex questions about malware binaries, with accuracies ranging from 69% for malware family classification to 98% for architecture classification. Our findings indicate that providing domain knowledge during training is essential for this application -- off-the-shelf models lack both accuracy and insight. We've deployed this emerging solution to a limited number of analysts to gather feedback for further improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。