构建开源数据集增强LLM生成硬件断言能力,效果超商用模型。
Enhancing Large Language Models for Hardware Verification: A Novel SystemVerilog Assertion Dataset
- 基于开源HDL代码合成带断言的SystemVerilog片段,系统化扩充数据。
- 微调后模型在多个平台性能超GPT-4o,最高提升96.88%。
- 支持本地训练,保障隐私且免授权费,适合学术与工业界使用。
硬件验证在现代SoC设计中至关重要,占开发时间约70%。SystemVerilog断言用于确保功能正确性,但现有工业实践依赖人工编写,随系统复杂度上升已难以为继。近年研究显示大语言模型(LLMs)可自动化此过程,但闭源先进模型如GPT-4o常生成错误断言且需昂贵许可,而小型开源模型需微调以应对HDL复杂性。为此,我们提出**VERT**——一个开源数据集,旨在通过增强开放源码HDL仓库中的变量,生成合成代码片段及其对应断言,助力开源模型微调。实验表明,微调后的Deepseek Coder 6.7B和Llama 3.1 8B在OpenTitan、CVA6、OpenPiton及Pulpissimo等平台表现超越GPT-4o,相较基线模型最高提升96.88%,较其高出24.14%。该数据集可私有本地微调,保障数据隐私并免除许可成本。项目地址:https://github.com/AnandMenon12/VERT。
原文摘要 · Abstract (English)
Hardware verification is crucial in modern SoC design, consuming around 70% of development time. SystemVerilog assertions ensure correct functionality. However, existing industrial practices rely on manual efforts for assertion generation, which becomes increasingly untenable as hardware systems become complex. Recent research shows that Large Language Models (LLMs) can automate this process. However, proprietary SOTA models like GPT-4o often generate inaccurate assertions and require expensive licenses, while smaller open-source LLMs need fine-tuning to manage HDL code complexities. To address these issues, we introduce **VERT**, an open-source dataset designed to enhance SystemVerilog assertion generation using LLMs. VERT enables researchers in academia and industry to fine-tune open-source models, outperforming larger proprietary ones in both accuracy and efficiency while ensuring data privacy through local fine-tuning and eliminating costly licenses. The dataset is curated by systematically augmenting variables from open-source HDL repositories to generate synthetic code snippets paired with corresponding assertions. Experimental results demonstrate that fine-tuned models like Deepseek Coder 6.7B and Llama 3.1 8B outperform GPT-4o, achieving up to 96.88% improvement over base models and 24.14% over GPT-4o on platforms including OpenTitan, CVA6, OpenPiton and Pulpissimo. VERT is available at https://github.com/AnandMenon12/VERT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。