arXiv:2412.11177cs.SEcs.LG2024-12被引 4

用渐进式训练提升二进制代码嵌入效果,兼顾通用性与任务适配。

A Progressive Transformer for Unifying Binary Code Embedding and Knowledge Transfer

  • 构建树状分层训练架构,知识从基础任务逐步流向专项任务。
  • 在7项二进制分析任务中平均性能提升14.8%,优于传统两阶段方法。
  • 无需复杂特征工程,适合处理加壳或去符号化代码的场景。

近年来,语言模型被引入二进制分析任务,如函数相似性检测和函数签名恢复。这些模型通常采用两阶段训练:先在机器码上通过掩码语言建模(MLM)预训练,再针对特定任务微调。尽管MLM有助于理解二进制结构,但忽略了控制流、数据流等关键特征,影响模型泛化能力。已有研究尝试在基于Transformer的方法中引入领域特定特征(如控制流图和动态执行轨迹),以增强语义理解,但需复杂的特征工程,过程繁琐且易引入预测不确定性,尤其在处理剥离或混淆代码时性能下降明显。本文提出ProTST,一种新型基于Transformer的二进制代码嵌入方法。ProTST采用基于独特树状结构的分层训练流程,知识从根部的基础任务逐步流向叶部的专项任务。该渐进式师生范式使模型能持续利用已有知识,生成高质量嵌入,可有效支持多种下游二进制分析任务。在7项二进制分析任务上的评估显示,ProTST相较传统两阶段训练平均得分提升14.8%,相比多模态两阶段框架平均提升10.7%。

原文摘要 · Abstract (English)

Language model approaches have recently been integrated into binary analysis tasks, such as function similarity detection and function signature recovery. These models typically employ a two-stage training process: pre-training via Masked Language Modeling (MLM) on machine code and fine-tuning for specific tasks. While MLM helps to understand binary code structures, it ignores essential code characteristics, including control and data flow, which negatively affect model generalization. Recent work leverages domain-specific features (e.g., control flow graphs and dynamic execution traces) in transformer-based approaches to improve binary code semantic understanding. However, this approach involves complex feature engineering, a cumbersome and time-consuming process that can introduce predictive uncertainty when dealing with stripped or obfuscated code, leading to a performance drop. In this paper, we introduce ProTST, a novel transformer-based methodology for binary code embedding. ProTST employs a hierarchical training process based on a unique tree-like structure, where knowledge progressively flows from fundamental tasks at the root to more specialized tasks at the leaves. This progressive teacher-student paradigm allows the model to build upon previously learned knowledge, resulting in high-quality embeddings that can be effectively leveraged for diverse downstream binary analysis tasks. The effectiveness of ProTST is evaluated in seven binary analysis tasks, and the results show that ProTST yields an average validation score (F1, MRR, and Recall@1) improvement of 14.8% compared to traditional two-stage training and an average validation score of 10.7% compared to multimodal two-stage frameworks.

二进制分析Transformer代码嵌入渐进学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。