无需训练,用错误笔记+检索增强实现3D CAD零件精准查找
Error Notebook-Guided, Training-Free Part Retrieval in 3D CAD Assemblies via Vision-Language Models
- 通过反思修正生成高质量错误笔记,构建规范的推理样本库
- 在人工偏好数据集上,GPT-4o准确率提升23.4个百分点
- 适合无法微调大模型的工程场景,尤其适用于开放模型
在复杂3D CAD装配体中实现基于规格的零件高效检索对自动化工程任务至关重要。然而,使用大语言模型或视觉语言模型(VLMs)面临挑战:CAD元数据序列常超令牌预算,且无法微调高性能专有模型(如GPT或Gemini)。为此,我们提出一种两阶段无训练框架,结合推理时自适应与检索增强生成(RAG),显著提升基于VLM的零件检索能力。每个错误笔记通过反思式修正初始思维链(CoT)生成,并经由提出的语法约束(GC)验证器筛选,确保结构正确性。最终形成的笔记库包含高质的规格-思维链-答案三元组,用于RAG检索相关示例以指导模型推理。我们还贡献了一个含人类偏好标注的CAD数据集。在专有模型(GPT-4o、Gemini等)上的实验表明,准确率最高提升23.4个百分点;GC验证器进一步带来最高4.5个百分点增益。该方法优于其他无训练基线(如少样本学习、自洽性),并对开源模型(Qwen2-VL-2B-Instruct、Aya-Vision-8B)也有显著提升。跨模型设置下,使用GPT-4o构建笔记,2B模型性能接近GPT-4o mini,差距仅约4分。
原文摘要 · Abstract (English)
Effective specification-aware part retrieval within complex CAD assemblies is essential for automated engineering tasks. However, using LLMs/VLMs for this task is challenging: the CAD model metadata sequences often exceed token budgets, and fine-tuning high-performing proprietary models (e.g., GPT or Gemini) is unavailable. Therefore, we need a framework that delivers engineering value by handling long, non-natural-language CAD model metadata using VLMs, but without training. We propose a 2-stage framework with inference-time adaptation that combines corrected Error Notebooks with RAG to substantially improve VLM-based part retrieval reasoning. Each Error Notebook is built by correcting initial CoTs through reflective refinement, and then filtering each trajectory using our proposed grammar-constraint (GC) verifier to ensure structural well-formedness. The resulting notebook forms a high-quality repository of specification-CoT-answer triplets, from which RAG retrieves specification-relevant exemplars to condition the model's inference. We additionally contribute a CAD dataset with human preference annotations. Experiments with proprietary models (GPT-4o, Gemini, etc) show large gains, with GPT-4o (Omni) achieving up to +23.4 absolute accuracy points on the human-preference benchmark. The proposed GC verifier can further produce up to +4.5 accuracy points. Our approach also surpasses other training-free baselines (standard few-shot learning, self-consistency) and yields substantial improvements also for open-source VLMs (Qwen2-VL-2B-Instruct, Aya-Vision-8B). Under the cross-model GC setting, where the Error Notebook is constructed using GPT-4o (Omni), the 2B model inference achieves performance that comes within roughly 4 points of GPT-4o mini.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。