arXiv:2601.13864cs.CRcs.AI2026-01被引 5

评测大模型生成硬件代码时的漏洞意识,发现其功能正确但常藏安全隐患。

HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation

  • 构建924个软硬件任务,覆盖76类常见漏洞,含可执行测试用例。
  • 多模型实测显示:功能达标率高,但安全缺陷普遍存在且与提示词有关。
  • 适合关注AI生成硬件代码安全性的研究者和开发者参考。

大型语言模型(LLMs)在硬件与固件代码生成中日益广泛应用,但现有研究主要评估功能正确性,严重忽视安全性。然而,看似功能正确的生成代码可能隐含安全缺陷,部署后可能导致灾难性后果。为此,我们设计了HardSecBench基准,包含924个任务,覆盖Verilog RTL与固件级C代码,涵盖76类硬件相关Common Weakness Enumeration(CWE)条目。每个任务配有结构化规格说明、安全参考实现及可执行测试。为自动化生成与验证,提出多智能体流水线,将合成与验证解耦,并基于执行证据进行评估,确保可靠性。实验评估多种LLMs发现,它们虽常满足功能要求,却普遍遗留安全风险,且结果受提示词影响显著。这些结果揭示了紧迫挑战,为未来基于LLM的硬件设计提供可行动洞察。数据与代码已开源于https://github.com/chenqirui2002/HardSecBench。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used for hardware and firmware code generation, but existing studies primarily evaluate functional correctness while largely overlooking security. However, LLM-generated code that appears functionally sound may embed security flaws which could induce catastrophic damages after deployment. This critical research gap motivates us to design a benchmark for assessing security awareness under realistic specifications. In this work, we introduce HardSecBench, a benchmark with 924 tasks spanning Verilog Register Transfer Level (RTL) and firmware-level C, covering 76 hardware-relevant Common Weakness Enumeration (CWE) entries. Each task includes a structured specification, a secure reference implementation, and executable tests. To automate artifact synthesis, we propose a multi-agent pipeline that decouples synthesis from verification and grounds evaluation in execution evidence, enabling reliable evaluation. We evaluate diverse LLMs and find that they often satisfy functional requirements while leaving security risks. We also find that security results vary with prompting. These findings highlight pressing challenges and offer actionable insights for future advancements in LLM-assisted hardware design. Our data and code are available at https://github.com/chenqirui2002/HardSecBench.

大模型安全硬件生成漏洞检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。