用强化学习提升大模型生成硬件测试用例的能力,准确率翻倍。
GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation
- 提出GRPO-SMu算法,通过状态变异增强模型探索能力。
- 7B模型测试通过率达33.3%,比基线提升17.6个百分点。
- 适合芯片设计自动化、AI辅助验证方向的研究者参考。
RTL设计早期常依赖非系统化的测试平台构建。尽管大语言模型在RTL代码生成方面展现潜力,但其对硬件规范的推理能力及生成针对性测试计划的能力仍不明确。本文首次系统研究了大模型在RTL验证激励生成中的推理能力,提出两阶段框架,将测试计划生成与测试平台执行解耦。基准测试显示,包括DeepSeek-R1和Claude-4.0-Sonnet在内的先进模型在生成能通过黄金RTL设计的激励时,成功率仅为15.7%-21.7%。为提升生成效果,我们开发了一种结合监督微调与新型强化学习方法的综合训练策略——GRPO with State Mutation(GRPO-SMu),通过输入变异增强探索能力。该方法采用基于树结构的分支变异策略,构建包含等价与变异树的训练数据,突破传统线性变异局限,提供更丰富的学习信号。在该数据集上训练的7B参数模型,实现33.3%的黄金测试通过率和13.9%的变异检测率,相比基线绝对提升17.6%,优于更大规模通用模型。结果表明,专项训练方法可显著提升大模型在硬件验证任务中的推理能力,为半导体设计流程中的自动化子单元测试奠定基础。
原文摘要 · Abstract (English)
RTL design often relies heavily on ad-hoc testbench creation early in the design cycle. While large language models (LLMs) show promise for RTL code generation, their ability to reason about hardware specifications and generate targeted test plans remains largely unexplored. We present the first systematic study of LLM reasoning capabilities for RTL verification stimuli generation, establishing a two-stage framework that decomposes test plan generation from testbench execution. Our benchmark reveals that state-of-the-art models, including DeepSeek-R1 and Claude-4.0-Sonnet, achieve only 15.7-21.7% success rates on generating stimuli that pass golden RTL designs. To improve LLM generated stimuli, we develop a comprehensive training methodology combining supervised fine-tuning with a novel reinforcement learning approach, GRPO with State Mutation (GRPO-SMu), which enhances exploration by varying input mutations. Our approach leverages a tree-based branching mutation strategy to construct training data comprising equivalent and mutated trees, moving beyond linear mutation approaches to provide rich learning signals. Training on this curated dataset, our 7B parameter model achieves a 33.3% golden test pass rate and a 13.9% mutation detection rate, representing a 17.6% absolute improvement over baseline and outperforming much larger general-purpose models. These results demonstrate that specialized training methodologies can significantly enhance LLM reasoning capabilities for hardware verification tasks, establishing a foundation for automated sub-unit testing in semiconductor design workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。