arXiv:2606.23697cs.SEcs.AI2026-06

用小模型精准切分C代码功能块,提升代码检索效率。

SemChunk-C: Semantic Segmentation for C Code

论文配图:SemChunk-C: Semantic Segmentation for C Code
图 1 · 摘自论文原文
  • 基于大模型语义理解,动态识别代码边界与功能类别。
  • 在多个真实代码数据集上边界准确率高,优于大型模型。
  • 轻量级模型适合嵌入式或资源受限的代码分析场景。

C类语言的代码语义切分仍具挑战,因语法复杂、宏展开及结构不规则。现有方法如固定窗口、启发式分割和语法工具常无法捕捉有意义的功能单元,影响下游大模型任务效果。本文提出一种面向C相关语言的语义切分方法:首先定义代码块类别;其次训练一个基于LLM的分类器,用于识别边界并赋予每段代码功能属性;通过大模型上下文理解能力实现灵活边界,适配具体结构。第三,提出SemChunk-C系列轻量级模型(17M~150M参数),基于前四个Ettin编码器,能有效识别数据结构、接口块等连贯代码单元。实验表明,该方法在包含嵌套定义和宏的复杂代码上表现稳健,在多个数据集上边界准确率高,语义一致性优,性能媲美甚至超越更大规模代码专用大模型。同时在若干精选基准上验证了下游任务性能提升。

原文摘要 · Abstract (English)

Semantic segmentation of code written in a C-family language remains a challenging problem, due to the language's complex syntax, macro expansion, and irregular structural patterns. Existing chunking methods, such as fixed-sized windows, heuristic splitting, and syntax-based tools, often fail to capture meaningful functional units, limiting the efficacy of retrieval and other downstream LLM driven tasks. In this paper, we address the problem of chunking in C-related languages. First, we define a set of code chunk categories. Second, we train an LLM-based classifier to a) identify chunk boundaries, and b) assign each chunk a descriptive functional attribute (a category), which can be useful for downstream tasks. By leveraging the LLM's ability to capture semantic context within the code, we assume flexible chunk boundaries, allowing to adapt to the specific structure and context of each instance. Third, we introduce SemChunk-C, a family of lightweight language models for semantic chunking of C-related files (.c, .cpp, .h, .cs, etc.). These models are based on the first four Ettin encoders [1] with 17M, 32M, 68M, and 150M parameters. Despite their relatively small size, they are capable of identifying cohesive code units, such as data structures, interface blocks, and other components. Furthermore, we demonstrate the robustness of our approach on real-world code, including challenging constructs such as nested definitions and macros. We test our approach on various datasets, and show that it achieves high boundary accuracy and semantic coherence, matching or outperforming chunkers that are based on much larger code-oriented LLMs. We also validate the improved performance of the downstream tasks on a few curated benchmarks.

代码理解语义切分轻量模型C语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。