用执行反馈生成测试用例，提升大模型生成芯片设计规则脚本的准确率。
Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation
- 基于执行结果生成测试用例，自动评估脚本正确性。
- 包含1000个任务和13921个布局，支持大规模真实场景评测。
- 适合芯片自动化设计、LLM脚本生成方向的研究者使用。
可制造的芯片版图需满足数千条基于几何的设计规则，设计规则检查（DRC）通过在版图上运行可执行的DRC脚本来强制执行这些规则。将自然语言规则转化为正确的DRC脚本既耗时又需专业知识，因此催生了基于大模型代理的DRC脚本生成与调试方法。然而，现有基准数据集规模小，且通常仅通过代码相似性评估脚本，而非执行正确性；先前的机器学习方法要么忽略执行反馈，要么需要标注的测试版图作为代理输入。为此，我们提出Rule2DRC，一个大规模的DRC脚本编码代理基准，包含1000个规则到脚本的任务及13,921个用于执行评分的芯片布局。Rule2DRC提供无需将评估版图作为输入的评估流程，通过DRC执行结果衡量功能正确性。我们还提出SplitTester，一种利用执行反馈生成判别性测试用例的测试代理，能有效区分此前难以区分的候选脚本，显著提升Best-of-N选择性能。代码已开源：https://github.com/snu-mllab/Rule2DRC。
原文摘要 · Abstract (English)
Manufacturable chip layouts must satisfy thousands of geometry-based design rules, and design rule checking (DRC) enforces them by running executable DRC scripts on layouts. Translating natural language rules into correct DRC scripts is labor-intensive and requires specialized expertise, motivating LLM agents for DRC script synthesis and debugging. However, existing benchmarks have small evaluation sets and often evaluate scripts by code similarity rather than execution correctness, and prior machine learning-based methods either ignore execution feedback or require labeled test layouts as agent's input. To this end, we introduce Rule2DRC, a large-scale benchmark for DRC script coding agents with 1,000 rule-to-script tasks and 13,921 evaluation chip layouts for execution-based scoring. Rule2DRC provides an evaluation pipeline that measures functional correctness via DRC execution outcomes without requiring evaluation layouts as input to the agent. We also propose SplitTester, a tester agent for program selection that uses execution feedback to generate discriminative test cases and separate previously indistinguishable candidate scripts, substantially improving Best-of-N selection performance in this domain. We release the code at https://github.com/snu-mllab/Rule2DRC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。