首个面向系统级电路图的多模态数据集与识别框架,显著提升非标准图纸理解能力。
DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams

- 构建分阶段多智能体流程,解耦感知、推理与知识三阶段视觉理解
- 在10,977个连接标注上,性能超越GPT-5等模型2倍以上
- 仅需60张图即可迁移适配,零样本推理媲美顶尖大模型
系统级电路图记录芯片设计的架构蓝图,包含模块功能、数据流与接口协议。然而,非标准化符号和缺乏结构化训练数据限制了现有多模态大模型对这类图的识别。为此,我们提出DiagramNet,首个面向系统级图的多模态数据集,涵盖10,977个连接标注和15,515个链式思维问答对,覆盖四类任务:列举、定位、连接与电路问答。基于该数据集,我们设计渐进式训练流程与解耦的多智能体工作流,将复杂视觉推理分解为感知、推理与知识三阶段。在DiagramNet基准上,集成3B参数模型与所提工作流,在端到端评估中超越2025年EDA精英挑战赛冠军,并优于GPT-5、Claude-Sonnet-4与Gemini-2.5-Pro超过2倍。值得注意的是,该工作流具有强泛化性,使Gemini-2.5-Pro在任务1上性能提升128.7倍,GPT-5提升12.4倍。此外,仅用60张图像微调检测器,方法即能有效迁移至AMSBench,实现零样本连接推理,表现媲美GPT-5与Claude-Sonnet-4,超越当前最先进方法Netlistify。
原文摘要 · Abstract (English)
System-level diagrams encode the architectural blueprint of chip design, specifying module functions, dataflows, and interface protocols. However, non-standardized symbols and the scarcity of structured training data hinder existing multimodal large language models (MLLMs) from recognizing these diagrams. To address this gap, we introduce DiagramNet, the first multimodal dataset for system-level diagrams, comprising 10,977 connection annotations and 15,515 chain-of-thought QA pairs across four tasks: Listing, Localization, Connection, and Circuit QA. Building on this dataset, we propose a progressive training pipeline together with a decoupled multi-agent workflow that decomposes complex visual reasoning into Perception, Reasoning, and Knowledge stages. On the DiagramNet benchmark, integrating our 3B-parameter model with the proposed workflow surpasses the 2025 EDA Elite Challenge winner and outperforms GPT-5, Claude-Sonnet-4, and Gemini-2.5-Pro by over 2x in end-to-end evaluation. Notably, the workflow generalizes beyond our model, boosting Task 1 performance by 128.7x for Gemini-2.5-Pro and 12.4x for GPT-5. Furthermore, with only 60 images for detector adaptation, the method transfers effectively to AMSBench, achieving zero-shot connectivity reasoning on par with GPT-5 and Claude-Sonnet-4 while surpassing the AMS state-of-the-art method Netlistify.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。