用强化学习+测试反馈提升Verilog代码生成的正确率和自检能力。
VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation
- 结合监督微调与奖励优化，利用测试用例反馈指导生成。
- 在VerilogEval上达成83.1%功能正确率，首试成功率提升2.8倍。
- 首次实现推理增强的Verilog生成，适合EDA自动化研究者。
使用大语言模型（LLM）自动化的寄存器传输级（RTL）代码生成有望显著简化数字电路设计并减少人工工作量。然而，现有基于LLM的方法面临训练数据稀缺、规格与代码对齐差、缺乏验证机制以及泛化与专业化难以平衡等问题。受DeepSeek-R1启发，我们提出VeriReason，一种结合监督微调与引导式奖励近端优化（GRPO）强化学习的RTL生成框架。通过精选训练样本和反馈驱动的奖励模型，VeriReason融合测试用例评估与结构启发式，并嵌入自检能力以实现自主纠错。在VerilogEval基准测试中，VeriReason取得显著提升：在VerilogEval Machine基准上达到83.1%的功能正确率，显著优于同等规模模型及更大型商业系统如GPT-4 Turbo。此外，本方法首试功能正确率相较基线提升最高达2.8倍，并展现出对未见设计的鲁棒泛化能力。据我们所知，VeriReason是首个成功将显式推理能力与强化学习结合用于Verilog生成的系统，为自动化RTL综合树立了新基准。模型与数据集已公开于：https://huggingface.co/collections/AI4EDA-CASE，代码地址：https://github.com/NellyW8/VeriReason。
原文摘要 · Abstract (English)
Automating Register Transfer Level (RTL) code generation using Large Language Models (LLMs) offers substantial promise for streamlining digital circuit design and reducing human effort. However, current LLM-based approaches face significant challenges with training data scarcity, poor specification-code alignment, lack of verification mechanisms, and balancing generalization with specialization. Inspired by DeepSeek-R1, we introduce VeriReason, a framework integrating supervised fine-tuning with Guided Reward Proximal Optimization (GRPO) reinforcement learning for RTL generation. Using curated training examples and a feedback-driven reward model, VeriReason combines testbench evaluations with structural heuristics while embedding self-checking capabilities for autonomous error correction. On the VerilogEval Benchmark, VeriReason delivers significant improvements: achieving 83.1% functional correctness on the VerilogEval Machine benchmark, substantially outperforming both comparable-sized models and much larger commercial systems like GPT-4 Turbo. Additionally, our approach demonstrates up to a 2.8X increase in first-attempt functional correctness compared to baseline methods and exhibits robust generalization to unseen designs. To our knowledge, VeriReason represents the first system to successfully integrate explicit reasoning capabilities with reinforcement learning for Verilog generation, establishing a new state-of-the-art for automated RTL synthesis. The models and datasets are available at: https://huggingface.co/collections/AI4EDA-CASE Code is Available at: https://github.com/NellyW8/VeriReason
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。