首个关注FPGA代码资源消耗的评测基准，区分LLM生成代码的优劣。
ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness
- 构建12类56个问题的资源感知评测集，覆盖从状态机到金融计算
- 通过自动验证与资源测量，发现不同LLM间LUT使用差异显著
- 适合硬件开发、LLM生成工具评估及资源优化研究者使用
现场可编程门阵列（FPGA）在现代硬件设计中广泛应用，但编写用于FPGA实现的硬件描述语言（HDL）代码仍是一项复杂且耗时的任务。大型语言模型（LLMs）已成为一种有前景的HDL生成工具，但现有的基于LLM的代码生成评测主要关注功能正确性，忽略了硬件资源使用情况。此外，现有评测数据集多样性不足，未能充分代表真实世界中的FPGA应用。为解决这些问题，我们提出ResBench，首个专为资源优化而设计的评测基准，旨在区分资源高效与低效的LLM生成的HDL代码。ResBench包含12个类别共56个问题，涵盖从有限状态机到金融计算的应用场景。我们的开源评估框架可自动生成Verilog代码、验证正确性并测量资源使用情况。实验以查找表（LUT）使用量为主要指标，结果表明不同LLM之间存在显著差异，证明了ResBench在识别生成更优资源利用率设计的模型方面的有效性。
原文摘要 · Abstract (English)
Field-Programmable Gate Arrays (FPGAs) are widely used in modern hardware design, yet writing Hardware Description Language (HDL) code for FPGA implementation remains a complex and time-consuming task. Large Language Models (LLMs) have emerged as a promising tool for HDL generation, but existing benchmarks for LLM-based code generation primarily focus on functional correctness while overlooking hardware resource usage. Furthermore, current benchmarks offer limited diversity and do not fully represent the wide range of real-world FPGA applications. To address these shortcomings, we introduce ResBench, the first resource-focused benchmark explicitly designed to distinguish between resource-optimized and inefficient LLM-generated HDL code. ResBench consists of 56 problems across 12 categories, covering applications from finite state machines to financial computing. Our open-source evaluation framework automatically tests LLMs by generating Verilog code, verifying correctness, and measuring resource usage. The experiments, which primarily analyze Lookup Table (LUT) usage, reveal significant differences among LLMs, demonstrating ResBench's capability to identify models that generate more resource-optimized FPGA designs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。