用强化学习让大模型生成更多样、更准确的硬件代码。
RTLSeek: Boosting the LLM-Based RTL Generation with Multi-Stage Diversity-Oriented Reinforcement Learning
- 分三阶段的强化学习，引导模型探索更多硬件设计可能。
- 在基准测试中生成质量优于现有方法，且越多越优。
- 适合想提升硬件设计多样性和正确性的工程师看。
寄存器传输级（RTL）设计将高层规格转化为Verilog等硬件描述语言。尽管基于大模型的RTL生成前景广阔，但功能可验证的高质量数据稀缺，限制了生成结果的准确性和多样性。现有后训练方法通常对每个规格只生成单一HDL实现，缺乏对不同设计目标所需变体的认知。本文提出RTLSeek，一种基于规则的多样性导向强化学习后训练范式，通过融合专家知识与EDA反馈，构建以多样性为中心的多目标奖励调度，并采用三阶段框架最大化有限数据的利用率。在RTLLM基准上的实验表明，RTLSeek超越已有方法；消融实验证实，鼓励更广泛的设计空间探索能显著提升RTL质量和“生成越多，效果越好”的原则。实现框架包括数据集、源代码和模型权重，详见https://anonymous.4open.science/r/DAC2026ID71-ACB4/。
原文摘要 · Abstract (English)
Register Transfer Level (RTL) design translates high-level specifications into hardware using HDLs such as Verilog. Although LLM-based RTL generation is promising, the scarcity of functionally verifiable high-quality data limits both accuracy and diversity. Existing post-training typically produces a single HDL implementation per specification, lacking awareness of RTL variations needed for different design goals. We propose RTLSeek, a post-training paradigm that applies rule-based Diversity-Oriented Reinforcement Learning to improve RTL correctness and diversity. Our Diversity-Centric Multi-Objective Reward Scheduling integrates expert knowledge with EDA feedback, and a three-stage framework maximizes the utility of limited data. Experiments on the RTLLM benchmark show that RTLSeek surpasses prior methods, with ablation results confirming that encouraging broader design-space exploration improves RTL quality and achieves the principle of "the more generated, the better results." Implementation framework, including the dataset, source code, and model weights, is shown at https://anonymous.4open.science/r/DAC2026ID71-ACB4/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。