提出新方法让AI真正理解电路图,而非靠名字猜代码
From Mirage to Grounding: Towards Reliable Multimodal Circuit-to-Verilog Code Generation

- 用匿名化标识符测试模型是否真看图,发现多数模型其实没看图
- 新模型VeriGround在无标识图下仍保持42.5%正确率,真实懂图
- 适合芯片设计、可信AI等对可靠性要求高的场景
多模态大模型被用于将视觉内容转为代码,如将电路图转为寄存器传输级(RTL)代码。我们发现一种新现象:用空白图替换电路图后,通过率(Pass@k)不降反升,因为模型依赖模块头中的标识符语义而非图像内容,这构成隐蔽的生成缺陷。为此我们构建了C2VEVAL数据集,采用配对的正常/匿名(Anony)测试协议,匿名化图中及模块头的所有标识符。结果显示,所有模型在匿名模式下准确率大幅下降,证明高正常模式表现实为“幻觉”。我们提出40亿参数的VeriGround模型,结合标识符匿名训练、拒绝增强和决策导向偏好对齐(D-ORPO),显著提升关键生成或拒绝判断能力。VeriGround在正常/匿名模式下功能通过率分别为46.11%/42.51%,错误拒绝率仅1.20%/0.00%,且对空白图保持超92%拒绝率,性能媲美GPT-5.4,证实其真正具备视觉接地能力。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) are increasingly used to translate visual artifacts into code, from UI mockups into HTML to scientific plots into Python scripts. A circuit diagram can be viewed as a visual domain-specific language for hardware: it encodes timing, topology, and bit level semantics that are invisible to casual inspection yet safety critical once fabricated in silicon. Translating such diagrams into register-transfer-level(RTL) code therefore represents an extreme reliability test for vision-to-code generation. We reveal a phenomenon we call Mirage: replacing a circuit diagram with a blank image leaves Pass@k unchanged or even higher, because models bypass the visual input and instead exploit identifier semantics in the module header to retrieve canonical RTL templates. This constitutes a new, highly covert class of defect in AI-assisted code generation that directly undermines MLLMs' trustworthiness. To quantify the effect, we construct C2VEVAL and evaluate eight MLLMs under a paired Normal/Anony protocol in which Anony mode anonymizes all identifiers in both the diagram and the module header; Anony-mode scores drop sharply across all models, confirming that high Normal-mode accuracy is largely a Mirage. We then propose VeriGround (4B), trained with identifier anonymization, refusal augmentation, and D-ORPO (Decision-Focused ORPO) preference alignment that up-weights pivotal generate-or-refuse tokens. VeriGround achieves Functional Pass@1 of 46.11%/42.51%(Normal/Anony) with a False Refusal Rate of only 1.20%/0.00%, while maintaining >92% Refusal Rate on blank images. With only 4B parameters, VeriGround performs on par with GPT-5.4 under Normal and significantly outperforms all baselines under Anony, confirming genuine visual grounding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。