arXiv:2512.07666cs.CLcs.SE2025-12被引 4

用外部模块让大模型读懂代码结构,提升生成与理解效果

Bridging Code Graphs and Large Language Models for Better Code Understanding

  • 通过可训练桥接模块融合代码图与文本信息
  • 代码摘要任务提升16.19%,翻译执行准确率提高38.87%
  • 无需修改模型架构,推理速度超LoRA方法4倍

大型语言模型(LLMs)在代码生成、摘要和翻译等任务中表现优异,但其依赖线性词元序列,难以理解程序的结构语义。现有方法或受提示长度限制,或需特定架构改动,不兼容大规模指令跟随式LLM。本文提出CGBridge,一种即插即用的方法,通过外部可训练的桥接模块,将代码图信息注入LLM。CGBridge首先在27万份代码图上进行自监督预训练,学习结构化代码语义;再训练桥接模块,通过跨模态注意力对齐代码、图与文本语义;最后生成结构感知提示注入冻结的LLM,并微调下游任务。实验表明,CGBridge在代码摘要任务中相对原模型提升16.19%(LLM-as-a-Judge),代码翻译执行准确率提升38.87%;且推理速度超过LoRA微调模型4倍,兼具高效与有效。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated remarkable performance in code intelligence tasks such as code generation, summarization, and translation. However, their reliance on linearized token sequences limits their ability to understand the structural semantics of programs. While prior studies have explored graphaugmented prompting and structure-aware pretraining, they either suffer from prompt length constraints or require task-specific architectural changes that are incompatible with large-scale instructionfollowing LLMs. To address these limitations, this paper proposes CGBridge, a novel plug-and-play method that enhances LLMs with Code Graph information through an external, trainable Bridge module. CGBridge first pre-trains a code graph encoder via selfsupervised learning on a large-scale dataset of 270K code graphs to learn structural code semantics. It then trains an external module to bridge the modality gap among code, graph, and text by aligning their semantics through cross-modal attention mechanisms. Finally, the bridge module generates structure-informed prompts, which are injected into a frozen LLM, and is fine-tuned for downstream code intelligence tasks. Experiments show that CGBridge achieves notable improvements over both the original model and the graphaugmented prompting method. Specifically, it yields a 16.19% and 9.12% relative gain in LLM-as-a-Judge on code summarization, and a 9.84% and 38.87% relative gain in Execution Accuracy on code translation. Moreover, CGBridge achieves over 4x faster inference than LoRA-tuned models, demonstrating both effectiveness and efficiency in structure-aware code understanding.

代码理解图神经网络大模型增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。