用功能正确性验证数据训练，提升大模型生成的硬件代码质量。
VeriCoder: Enhancing LLM-Based RTL Code Generation through Functional Correctness Validation
- 通过测试生成与反馈迭代，构建功能正确的硬件代码数据集。
- 在125,777个样本上训练，功能正确率比现有方法最高提升71.7%。
- 适合需要高可靠性硬件生成的工程师或研究者使用。
大型语言模型（LLM）在电子设计自动化（EDA）任务中日益受到关注，尤其是在寄存器传输级（RTL）代码生成方面。尽管已有多个RTL数据集发布，但多数仅关注语法正确性，未进行功能验证，导致训练样本虽能编译却可能无法实现预期行为。本文提出VERICODER，一个基于功能正确性验证数据集微调的RTL代码生成模型。该数据集采用新方法构建：给定自然语言规格和初始RTL设计，利用教师模型（GPT-4o-mini）生成单元测试，并根据仿真结果迭代优化RTL设计；必要时，教师模型还更新测试以确保其符合自然语言描述。最终每个样本均包含通过测试的自然语言说明、RTL实现与测试用例。在包含125,777个示例的数据集上微调后，VERICODER在VerilogEval和RTLLM上的功能正确率分别达到71.7%和27.4%的相对提升，显著优于现有方法。消融实验进一步证明，基于功能验证数据训练的模型优于仅依赖语法合法性的模型，凸显高质量数据集对硬件生成的重要性。代码、数据与模型已开源。
原文摘要 · Abstract (English)
Recent advances in Large Language Models (LLMs) have sparked growing interest in applying them to Electronic Design Automation (EDA) tasks, particularly Register Transfer Level (RTL) code generation. While several RTL datasets have been introduced, most focus on syntactic validity rather than functional validation with tests, leading to training examples that compile but may not implement the intended behavior. We present VERICODER, a model for RTL code generation fine-tuned on a dataset validated for functional correctness. This fine-tuning dataset is constructed using a novel methodology that combines unit test generation with feedback-directed refinement. Given a natural language specification and an initial RTL design, we prompt a teacher model (GPT-4o-mini) to generate unit tests and iteratively revise the RTL design based on its simulation results using the generated tests. If necessary, the teacher model also updates the tests to ensure they comply with the natural language specification. As a result of this process, every example in our dataset is functionally validated, consisting of a natural language description, an RTL implementation, and passing tests. Fine-tuned on this dataset of 125,777 examples, VERICODER achieves state-of-the-art metrics in functional correctness on VerilogEval and RTLLM, with relative gains of up to 71.7% and 27.4%, respectively. An ablation study further shows that models trained on our functionally validated dataset outperform those trained on functionally non-validated datasets, underscoring the importance of high-quality datasets in RTL code generation. Our code, data, and models are publicly available at https://github.com/Anjiang-Wei/VeriCoder
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。