用构造正确数据提升Verilog代码生成质量，修复模型常见错误
CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair
- 构建可正确构造的非文本表示数据，解决硬件描述难题
- 通过注入错误生成修复数据，模型性能提升3.8%至10.9%
- 适合芯片设计与自动化验证研究者使用
尽管大型语言模型在代码生成方面取得进展，但在硬件描述语言如Verilog上仍面临挑战。本文分析了微调后的LLM在Verilog编码中的表现，发现两大问题：难以处理卡诺图、状态转移图和波形等非文本表示；训练中模型随机出现‘微小’错误导致结果波动。为此，我们通过构建正确性保障的数据来优化数据采集，并提出自动化框架，从不同模型检查点生成错误报告，注入开源代码以创建针对性修复数据。基于此，微调后的Sorcoder2-15B在VerilogEval-Machine、VerilogEval-Human和RTLLM上的pass@1分别提升3.8%、10.9%、6.6%，超越现有最佳结果。
原文摘要 · Abstract (English)
Despite the significant progress made in code generation with large language models, challenges persist, especially with hardware description languages such as Verilog. This paper first presents an analysis of fine-tuned LLMs on Verilog coding, with synthetic data from prior methods. We identify two main issues: difficulties in handling non-textual representations (Karnaugh maps, state-transition diagrams and waveforms) and significant variability during training with models randomly making "minor" mistakes. To address these limitations, we enhance data curation by creating correct-by-construction data targeting non-textual representations. Additionally, we introduce an automated framework that generates error reports from various model checkpoints and injects these errors into open-source code to create targeted code repair data. Our fine-tuned Starcoder2-15B outperforms prior state-of-the-art results by 3.8%, 10.9%, 6.6% for pass@1 on VerilogEval-Machine, VerilogEval-Human, and RTLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。