arXiv:2605.23931cs.AIcs.PL2026-05

用领域知识引导大模型精准生成操作系统内核规范。

BODHI: Precise OS Kernel Specification Inference

  • 设计结构化提示模板,分步骤提取前提与后置条件。
  • 在245个任务上将准确率从55.1%提升至96.7%。
  • 适合需要高可靠内核验证的研究者与工程师。

操作系统内核的形式化验证依赖于精确的系统调用行为规范。手动编写这些规范需深厚领域知识,因此我们提出利用大语言模型(LLMs)自动化生成。在基于Hyperkernel内核的OSV-Bench基准测试中,现有最佳方法的Pass@1仅为55.10%。本文提出一种领域知识提示方法(BODHI),在标准少样本提示基础上,加入涵盖15类领域特异性转换模式的结构化C-to-Python翻译指南。受结构化思维链(SCoT)启发,指南按关注点分离原则组织,分别处理前置条件提取与后置条件生成。在六家厂商(Anthropic、Mistral、Amazon、DeepSeek、Meta、Alibaba)的九种模型上评估,覆盖密集型、专家混合及推理架构,所有模型均获提升,增益达+11%至+32%。最优配置(Claude Opus 4.6 + BODHI)达到96.73% Pass@1。BODHI同时减少语法与语义错误,尤其对具备较强指令遵循能力的模型效果显著。结果表明,领域知识注入是一种模型无关的技术,能有效弥合通用代码生成与形式化规范合成之间的差距。

原文摘要 · Abstract (English)

The formal verification of operating system kernels requires precise specifications that capture the intended behavior of system calls. Writing these specifications manually demands deep domain expertise, motivating the use of large language models (LLMs) to automate the process. However, in OSV-Bench, a benchmark of 245 specification generation tasks derived from the Hyperkernel OS kernel, the best reported Pass@1 is 55.10%. We propose a domain knowledge prompting method (BODHI), which augments the standard few-shot prompt with a structured C-to-Python translation guide covering 15 categories of domain-specific translation patterns. Inspired by Structured Chain-of-Thought (SCoT) prompting, the guide organizes translation by separation of concerns, addressing pre-condition extraction and post-condition generation as distinct categories. Evaluated on nine models from six providers (Anthropic, Mistral, Amazon, DeepSeek, Meta, Alibaba), covering dense, mixture-of-experts and reasoning architectures, BODHI improves every model tested, with gains ranging from +11% to +32%. The best configuration (Claude Opus 4.6 + BODHI) reaches 96.73% Pass@1. BODHI reduces both syntax and semantic errors, with the strongest effect on models that have sufficient instruction-following capability to utilize structured reference material. These results demonstrate that domain knowledge injection is a model-agnostic technique that substantially bridges the gap between general-purpose code generation and formal specification synthesis.

形式化验证大模型内核规范

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。