研究大模型与提示词在Verilog代码生成中的互动关系
VeriInteresting: An Empirical Study of Model Prompt Interactions in Verilog Code Generation
- 对比多种模型与提示工程策略的组合效果
- 发现结构化提示能显著提升生成质量，尤其对小模型
- 适合关注硬件设计自动化与提示优化的研究者
语言模型（LMs）的快速发展为代码自动生成带来新机遇，但也加剧了模型特性与提示设计之间的权衡。本文针对最近在Verilog代码生成中的语言模型趋势，实证分析模型推理能力、专业化程度与提示工程策略之间的交互关系。评估涵盖通用、推理型及领域专用等多种小型和大型语言模型。实验采用受控因子设计，覆盖基准提示、结构化输出、提示重写、思维链推理、上下文学习以及基于遗传-帕累托算法的进化式提示优化。在两个Verilog基准测试中，识别出不同模型类别对结构化提示与优化策略的响应模式，并记录了哪些趋势在不同模型与基准间具有普适性，哪些仅存在于特定模型-提示组合中。
原文摘要 · Abstract (English)
Rapid advances in language models (LMs) have created new opportunities for automated code generation while complicating trade-offs between model characteristics and prompt design choices. In this work, we provide an empirical map of recent trends in LMs for Verilog code generation, focusing on interactions among model reasoning, specialization, and prompt engineering strategies. We evaluate a diverse set of small and large LMs, including general-purpose, reasoning, and domain-specific variants. Our experiments use a controlled factorial design spanning benchmark prompts, structured outputs, prompt rewriting, chain-of-thought reasoning, in-context learning, and evolutionary prompt optimization via Genetic-Pareto. Across two Verilog benchmarks, we identify patterns in how model classes respond to structured prompts and optimization, and we document which trends generalize across LMs and benchmarks versus those that are specific to particular model-prompt combinations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。