让代码模型同时理解文本与结构信息，提升代码推理能力。
Modeling Code: Is Text All You Need?
- 将代码文本与结构化数据结合建模
- 兼顾生成能力与分析性能
- 适合需要深度代码理解的任务
近年来，基于Transformer的代码大模型在代码生成、翻译和摘要等任务中广泛应用。然而，这类模型在处理代码的控制流、数据流等结构性分析特性时能力有限。此前研究尝试通过结构化数据和图神经网络建模这些特性，但缺乏现代大模型的生成能力与规模优势。本文提出一种新方法，融合代码的文本表示与更结构化的形式，以同时发挥两者优势。
原文摘要 · Abstract (English)
Code LLMs have become extremely popular recently for modeling source code across a variety of tasks, such as generation, translation, and summarization. However, transformer-based models are limited in their capabilities to reason through structured, analytical properties of code, such as control and data flow. Previous work has explored the modeling of these properties with structured data and graph neural networks. However, these approaches lack the generative capabilities and scale of modern LLMs. In this work, we introduce a novel approach to combine the strengths of modeling both code as text and more structured forms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。