用几何反馈自动生成高质量CAD训练数据,提升图像转程序的准确率
GIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback
- 通过测试时几何反馈生成新训练样本,无需人工标注
- 比强基线模型平均交并比提升12%,推理计算量降低80%
- 适合想构建鲁棒生成式CAD系统的研究人员
从图像生成可执行的CAD程序需要视觉几何与符号程序表示之间的对齐,但现有方法在设计复杂度增加时难以可靠学习。当前微调方法依赖有限的监督数据集或昂贵的后训练流程,导致系统脆弱,限制了生成式CAD的发展。我们指出主要瓶颈并非模型或算法能力,而是视觉几何与程序语法对齐的多样化训练样本稀缺。由于多样且验证过的工程数据集收集成本高、难扩展,制约了鲁棒生成式CAD模型的发展。本文提出几何推断反馈调优(GIFT),一种利用几何反馈将测试时计算转化为高质量训练样本的数据增强框架。GIFT结合两种机制:软拒绝采样(GIFT-REJECT)保留超出精确真值匹配的多样化高保真程序;失败驱动增强(GIFT-FAIL)将近似预测转为合成训练样本,提升对复杂几何的鲁棒性。通过将推理时搜索成本分摊到模型参数中,GIFT在减少80%推理计算的同时,实现测试时扩展收益。相比强监督基线,平均交并比提升12%,且不需额外人工标注或专用架构,性能媲美更复杂的多模态系统。
原文摘要 · Abstract (English)
Generating executable CAD programs from images requires alignment between visual geometry and symbolic program representations, a capability that current methods fail to learn reliably as design complexity increases. Existing fine-tuning approaches rely on either limited supervised datasets or expensive post-training pipelines, resulting in brittle systems that restrict progress in generative CAD design. We argue that the primary bottleneck lies not in model or algorithmic capacity, but in the scarcity of diverse training examples that align visual geometry with program syntax. This limitation is especially acute because the collection of diverse and verified engineering datasets is both expensive and difficult to scale, constraining the development of robust generative CAD models. We introduce Geometric Inference Feedback Tuning (GIFT), a data augmentation framework that leverages geometric feedback to turn test-time compute into a bootstrapped set of high-quality training samples. GIFT combines two mechanisms: Soft-Rejection Sampling (GIFT-REJECT), which retains diverse high-fidelity programs beyond exact ground-truth matches, and Failure-Driven Augmentation (GIFT-FAIL), which converts near-miss predictions into synthetic training examples that improve robustness on challenging geometries. By amortizing inference-time search into the model parameters, GIFT captures the benefits of test-time scaling while reducing inference compute by 80%. It improves mean IoU by 12% over a strong supervised baseline and remains competitive with more complex multimodal systems, without requiring additional human annotation or specialized architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。