用AI自动生成可验证代码,解决形式化验证数据稀缺问题。
Formal Disco: Scalable Open-Ended Generation of Formally Verified Programs

- 设计分布式系统协调多类AI工人生成代码
- 在三种语言中生成大量可验证程序,性能媲美顶级模型
- 适合形式化验证、AI编程与自动推理研究者
随着AI助手生成代码的成本不断降低,代码质量保障却未能同步提升。形式化验证虽能提供最强保证,但因相关语言的高质量人工示例稀少,AI模型难以有效学习。为此,我们提出Formal Disco:一种基于LLM的分布式系统,可规模化生成开放式的合成数据。该系统通过三类工人协作:‘发起者’从开源仓库读取文档草拟验证程序,‘修复者’根据编译与验证反馈修正错误,‘扩展者’对已运行程序提出改进补丁。系统记录所有生成过程,用于从强模型蒸馏知识并实现自我优化。我们还提出最大熵原则,通过迭代监督微调实现生成多样性持续提升。我们发布了涵盖Dafny、Verus和Frama-C三种语言的大规模合成验证程序数据集,并微调了开源模型,使其在验证相关任务上的表现通常达到甚至超过Claude Opus 4.5。本工作为形式化推理领域提供了规模化合成数据生成路径,突破长期存在的数据瓶颈。
原文摘要 · Abstract (English)
The cost of producing code is rapidly diminishing with increasingly capable AI agents, while quality assurance of generated programs has not kept pace. Formal verification provides the strongest possible guarantees, but the ability of AI models to work with verification-aware languages is hindered by the scarcity of human-written examples of programs in those languages. To tackle this prevalent data scarcity issue, we propose Formal Disco: a distributed system for coordination of LLM-based workers that can be easily applied to open-ended synthetic data generation at scale. We use Formal Disco to share tasks and programs between three classes of workers: "initiators", which read random READMEs from open-source repositories and documentation snippets to sketch a related verified program, "fixers" which take compiler and verifier feedback and attempt to resolve issues, and "extenders" that take working programs and propose patches to expand them. Formal Disco records all agent-generated traces and uses them both for initial distillation from a stronger model as well as self-improvement. We also propose a principle of maximum entropy for synthetic program generation, and use entropy maximization via iterative supervised fine-tuning to learn to generate increasingly diverse programs over time. We release large datasets of synthetic verified programs in three languages - Dafny, Verus, and Frama-C -, and fine-tune open models for verification-relevant tasks, often matching or exceeding the performance of Claude Opus 4.5. Overall, our work offers a path to create synthetic data at scale for formal reasoning domains and overcome the long-standing data barrier.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。