arXiv:2503.11617cs.SEcs.AI2025-03被引 5

用结构语义指令微调,让大模型读懂汇编代码。

ASMA-Tune: Unlocking LLMs' Assembly Code Comprehension via Structural-Semantic Instruction Tuning

  • 用编码器提取汇编结构特征,通过投影模块连通语义空间。
  • 在汇编理解任务上超越GPT-4-Turbo,Recall@1提升39.7%。
  • 适合做逆向工程、二进制分析的开发者和研究人员。

汇编代码分析与理解在逆向工程等应用中至关重要,但因信息密度低且缺乏显式语法结构而面临挑战。传统掩码语言建模(MLM)方法不关注自然语言交互,而新兴的解码器型大语言模型(LLMs)虽在二进制分析中表现部分成功,却未被充分探索用于整体理解。本文提出装配增强微调(ASMA-Tune),一种端到端的结构-语义指令微调框架,通过投影模块将编码器架构与基于解码器的LLM协同,其中汇编编码器提取硬件级结构特征,投影模块桥接表示至语义空间,指令微调后的LLM保留自然语言能力。实验表明三大优势:(1) 汇编理解性能达到当前最优,相比GPT-4-Turbo Recall@1提升39.7%,MRR提升17.8%;(2) 在多个基础模型(Qwen2.5-Coder、Deepseek-Coder、CodeLlama系列)上均实现24.6%-107.4%的Recall@1提升和15.2%-106.3%的MRR提升;(3) 指令遵循能力显著增强(提升41.5%-118%),代码生成质量仅轻微下降(-8.9%至-35%)。

原文摘要 · Abstract (English)

Assembly code analysis and comprehension play critical roles in applications like reverse engineering, yet they face substantial challenges due to low information density and a lack of explicit syntactic structures. While traditional masked language modeling (MLM) approaches do not explicitly focus on natural language interaction, emerging decoder-focused large language models (LLMs) demonstrate partial success in binary analysis yet remain underexplored for holistic comprehension. We present Assembly Augmented Tuning, an end-to-end structural-semantic instruction tuning framework that synergizes encoder architecture with decoder-based LLMs through a projector module, where the assembly encoder extracts hardware-level structural features, the projector bridges representations with the semantic space, and the instruction-tuned LLM preserves natural language capabilities. Experimental results demonstrate three key advantages: (1) State-of-the-art performance in assembly comprehension with +39.7% Recall@1 and +17.8% MRR improvements over GPT-4-Turbo, (2) Consistent enhancements across base models (24.6-107.4% Recall@1 and 15.2-106.3% MRR on Qwen2.5-Coder, Deepseek-Coder and CodeLlama variants), and (3) Superior instruction-following capabilities (41.5%-118% improvements) with controlled code generation degradation (-8.9% to -35% across architectures).

汇编理解大模型逆向工程指令微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。