用场景化框架让大模型生成可编译的C单元测试
SPARC: Scenario Planning and Reasoning for Automated C Unit Test Generation
- 分四步走:分析控制流、用工具函数锚定推理、路径靶向生成、编译运行反馈迭代修复
- 在59个真实项目上,分支覆盖率比基线高26.01%,接近KLEE工具水平
- 自动生成的测试94.3%能通过修复,代码可读性获开发者好评
C语言自动化单元测试仍面临巨大挑战,源于高层次程序意图与指针运算、手动内存管理等语法约束之间的语义鸿沟。尽管大语言模型(LLM)具备强大生成能力,但直接从意图生成代码常出现‘跳跃式编码’错误,导致无法编译的测试、虚构的函数签名、低分支覆盖率和无关的断言,无法有效捕捉缺陷。本文提出SPARC,一种神经符号式、基于场景的框架,通过四个阶段实现突破:(1) 控制流图(CFG)分析,(2) 使用操作映射将LLM推理锚定在经验证的实用辅助函数上,(3) 路径靶向测试生成,(4) 借助编译器与运行时反馈的迭代自校正验证循环。我们在59个真实与算法类项目上评估,相比原始提示基线,线覆盖提升31.36%,分支覆盖提升26.01%,突变得分提升20.78%,在复杂任务上达到或超越符号执行工具KLEE的表现。SPARC通过迭代修复保留了94.3%的测试,生成代码显著提升开发者评价的可读性与可维护性。该方法通过对齐大模型推理与程序结构,为工业级遗留C代码测试提供了可扩展路径。
原文摘要 · Abstract (English)
Automated unit test generation for C remains a formidable challenge due to the semantic gap between high-level program intent and the rigid syntactic constraints of pointer arithmetic and manual memory management. While Large Language Models (LLMs) exhibit strong generative capabilities, direct intent-to-code synthesis frequently suffers from the leap-to-code failure mode, where models prematurely emit code without grounding in program structure, constraints, and semantics. This will result in non-compilable tests, hallucinated function signatures, low branch coverage, and semantically irrelevant assertions that cannot properly capture bugs. We introduce SPARC, a neuro-symbolic, scenario-based framework that bridges this gap through four stages: (1) Control Flow Graph (CFG) analysis, (2) an Operation Map that grounds LLM reasoning in validated utility helpers, (3) Path-targeted test synthesis, and (4) an iterative, self-correction validation loop using compiler and runtime feedback. We evaluate SPARC on 59 real-world and algorithmic subjects, where it outperforms the vanilla prompt generation baseline by 31.36% in line coverage, 26.01% in branch coverage, and 20.78% in mutation score, matching or exceeding the symbolic execution tool KLEE on complex subjects. SPARC retains 94.3% of tests through iterative repair and produces code with significantly higher developer-rated readability and maintainability. By aligning LLM reasoning with program structure, SPARC provides a scalable path for industrial-grade testing of legacy C codebases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。