用检索增强与合成微调提升大模型生成硬件断言的能力
Hybrid-NL2SVA: Integrating RAG and Finetuning for LLM-based NL2SVA
- 结合检索增强与合成数据微调,改进大模型理解硬件断言语法语义
- 相比GPT-4o-mini,功能匹配断言数提升58.42%
- 适合硬件验证工程师和大模型应用开发者参考
系统级硬件断言(SVAs)对验证硬件设计正确性至关重要,但将自然语言描述自动转为SVA仍依赖人工,效率低且易出错。近期大语言模型(LLMs)为该任务带来新可能,但现有模型仍难以掌握领域特有语法与语义。为此,我们提出定制化的检索增强生成(RAG)框架与合成微调数据集,共同提升LLM在NL2SVA任务中的表现。针对轻量级模型,微调数据集提供逐层引导的解释,教会模型如何构建并发式SVA,实现监督微调,显著提升语法与功能准确性。为评估性能,我们构建了目前最大的NL2SVA评测数据集,包含40个Verilog设计和229条经形式化验证的SVA,并附详细标注。实验表明,所提定制RAG框架使功能匹配的SVA数量比GPT-4o-mini高出58.42%;而基于Qwen2.5-Coder-7B-Instruct的模型在本数据集上微调并融合HybridRetrieval后,相较原版模型提升59.05%。
原文摘要 · Abstract (English)
SystemVerilog Assertions (SVAs) are critical for verifying the correctness of hardware designs, but manually writing them from natural language property descriptions, i.e., NL2SVA, remains a labor-intensive and error-prone task. Recent advances in large language models (LLMs) offer opportunities to automate this translation. However, existing models still struggle with understanding domain-specific syntax and semantics. To enhance LLM performance in NL2SVA, we propose a customized retrieval-augmented generation (RAG) framework and a synthetic fine-tuning dataset that together improve LLM's performance. To further improve lightweight models over NL2SVA, our fine-tuning dataset provides prompt-guided explanations that teach LLMs the layer-by-layer construction process of concurrent SVAs, enabling supervised fine-tuning that greatly improves syntax and functionality accuracy. To evaluate the performance of LLMs over NL2SVA, we construct the largest evaluation dataset for NL2SVA, comprising 40 Verilog designs and 229 formally verified SVAs with detailed annotations. Experimental results show that our customized RAG framework increases the number of functionality matched SVAs by 58.42% over GPT-4o-mini, while Qwen2.5-Coder-7B-Instruct fine-tuned on our fine-tuning dataset and integrated with HybridRetrieval achieves a 59.05% over the base Qwen model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。