arXiv:2608.13459cs.SEcs.AI2026-08被引 1

用可验证契约修复Isabelle证明,防止LLM越权修改代码。

CAPRI: Contract-Aware Proof Repair for Isabelle

  • 引入契约机制,确保LLM只在授权范围内修改证明。
  • 36次修复中仅6次违规,证明体接口零违规且修复成功率97%。
  • 适合需高安全性的形式化验证场景,尤其关注自动化修复的可靠性。

我们研究如何利用大语言模型(LLMs)辅助发现Isabelle证明。尽管构建过程能确认理论被接受,但无法保证LLM仅修改了开发者授权的内容。为此提出CAPRI,一种合同感知的修复流程:Isabelle检查证明,独立校验器强制执行机器可读的编辑合同。所有提示、建议、候选库、诊断、裁决和哈希均保留用于审计。在四个项目中的12个失败证明上评估五种工作流,每项任务与条件重复三次,共180次运行,获得138次有效修复。144个由Isabelle接受的最终候选中,6个修改了受保护文本,均来自可编辑完整理论的迭代工作流。仅限证明体的接口实现29/36次有效修复且无合同违规,优于对应全理论工作流的31/36。一次性修复为22/36,后期前瞻性冻结的迭代工作流达32/36;这些数字反映整体工作流表现。另一次事后使用OpenRouter的测试未在指定的Luna对比中提升效果。匹配示例的Sol配置实现33/36修复,优于冻结的OpenAI响应条件(29/36),但单侧精确麦克内马尔检验显示差异不显著(p=0.0625)。

原文摘要 · Abstract (English)

We address the use of large language models (LLMs) to help discover Isabelle proofs. An Isabelle build establishes that the submitted theory is accepted, but not that an LLM changed only what the developer authorised. We present CAPRI, a contract-aware repair workflow in which Isabelle checks the proof and an independent checker enforces a machine-readable edit contract. Prompts, proposals, candidate repositories, diagnostics, verdicts, and hashes are retained for audit. We evaluate five workflows on twelve failed proofs from four developments, with three replicates per task and condition, giving 180 runs and 138 valid repairs. Of 144 terminal candidates accepted by Isabelle, six had modified protected text; all arose in iterative workflows that could edit a complete theory. A proof-body-only interface produced 29/36 valid repairs and no contract violations, compared with 31/36 for the corresponding full-theory workflow. One-shot repair produced 22/36, while a later prospectively frozen iterative workflow produced 32/36; these figures compare complete workflows rather than individual mechanisms. A separate post hoc OpenRouter campaign found no improvement in the designated Luna comparisons. A Sol configuration with matched demonstrations produced 33/36 repairs, compared with 29/36 in the frozen OpenAI Responses condition, but the difference was not statistically significant in a one-sided exact McNemar test ($p=0.0625$).

形式化验证LLM修复Isabelle安全合约

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。