通过信号级奖励优化,提升硬件代码生成的正确率。
QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation
- 用信号级正确性判断替代模块级验证,提取有效奖励信号。
- 在70亿参数下达到6710亿参数模型的性能水平。
- 适合需要高可靠性硬件设计的AI辅助开发人员。
大型语言模型在硬件电路自动设计中的Verilog代码生成方面展现出巨大潜力。然而,缺乏有意义的功能奖励制约了基于强化学习的偏好优化,难以生成功能正确的代码。本文提出信号感知学习方法QiMeng-SALV,利用训练数据中功能正确输出信号的代码片段,优化强化学习训练过程。由于Verilog代码描述硬件门与线的结构连接,不同输出信号相互独立,关键思想是提取部分错误模块中经验证的信号级实现,以增强有意义的功能奖励提取。具体而言,通过对比生成模块与参考模块的信号功能正确性进行验证;再使用抽象语法树(AST)识别可提供有效奖励的信号级代码段;最后引入信号级直接偏好优化(DPO),仅在正确信号级代码段上优化,从而避免错误信号带来的噪声干扰。实验表明,该方法在VerilogEval和RTLLM基准上达到当前最优性能,70亿参数模型表现媲美6710亿参数的DeepSeek v3,并显著优于同数据集训练的开源领先模型CodeV。代码已开源:https://github.com/QiMeng-IPRC/QiMeng-SALV。
原文摘要 · Abstract (English)
The remarkable progress of Large Language Models (LLMs) presents promising opportunities for Verilog code generation which is significantly important for automated circuit design. The lacking of meaningful functional rewards hinders the preference optimization based on Reinforcement Learning (RL) for producing functionally correct Verilog code. In this paper, we propose Signal-Aware Learning for Verilog code generation (QiMeng-SALV) by leveraging code segments of functionally correct output signal to optimize RL training. Considering Verilog code specifies the structural interconnection of hardware gates and wires so that different output signals are independent, the key insight of QiMeng-SALV is to extract verified signal-aware implementations in partially incorrect modules, so as to enhance the extraction of meaningful functional rewards. Roughly, we verify the functional correctness of signals in generated module by comparing with that of reference module in the training data. Then abstract syntax tree (AST) is employed to identify signal-aware code segments which can provide meaningful functional rewards from erroneous modules. Finally, we introduce signal-aware DPO which is optimized on the correct signal-level code segments, thereby preventing noise and interference from incorrect signals. The proposed QiMeng-SALV underscores the paradigm shift from conventional module-level to fine-grained signal-level optimization in Verilog code generation, addressing the issue of insufficient functional rewards. Experiments demonstrate that our method achieves state-of-the-art performance on VerilogEval and RTLLM, with a 7B parameter model matching the performance of the DeepSeek v3 671B model and significantly outperforming the leading open-source model CodeV trained on the same dataset. Our code is available at https://github.com/QiMeng-IPRC/QiMeng-SALV.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。