首个将CAD的边界表示与多模态语言图像对齐的预训练框架
BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding
- 用面和边的离散符号序列建模CAD几何，融合空间与语义描述符
- 在ABC、CADParser等数据集上文本检索准确率提升超20%，零样本分类增15%
- 适合做结构感知的CAD生成评估，也适用于多模态设计理解任务
学习CAD模型的表征仍是开放问题。尽管点云和网格的3D表征学习已蓬勃发展，但CAD原生格式——边界表示（BRep）所蕴含的精确参数曲面、曲线及其拓扑关系，却长期未被重视。本文提出BRepCLIP，首个通过对比预训练将BRep几何与语言、图像嵌入对齐的框架。将每个CAD对象建模为面与边的符号序列，分别使用独立词汇表表示表面与曲线几何，并加入捕捉表面类型（如圆柱、环面、NURBS）和曲线原型（如直线、弧、B样条）的空间与语义描述符。通过Transformer编码器聚合这些符号，生成全局BRep嵌入，并通过联合对比目标与CLIP的文本和图像编码器对齐。相比现有基于点的方法，BRepCLIP生成的嵌入更具区分性且语义更扎实，在ABC、CADParser、Automate数据集上的Top-1检索准确率分别提升40.4%、22.0%和23.9%；在FabWave数据集上零样本分类的Top-1得分提高15%。进一步验证其作为多模态条件下的CAD生成评估指标的有效性，凸显结构感知预训练在多模态CAD理解中的重要性。
原文摘要 · Abstract (English)
Learning representations of CAD models is a largely open problem. While 3D representation learning has flourished around point clouds and meshes, the native format of CAD - boundary representations BReps, which encodes exact parametric surfaces, curves, and their topology, has received little attention as a representation learning substrate. We introduce BRepCLIP, the first framework to align BRep geometry with language and image embeddings through contrastive pretraining. We model each CAD object as a sequence of face and edge tokens with separate discrete vocabularies for surface and curve geometry, augmented with spatial and semantic descriptors that capture surface types (e.g., cylindrical, torus, NURBS) and curve primitives (e.g., line, arc, B-spline). A transformer encoder aggregates these tokens into a global BRep embedding, aligned with CLIP's text and image encoders via a joint contrastive objective. BRepCLIP generates more discriminative and semantically grounded embeddings than existing point-based alternatives, improving Top-1 retrieval over OpenShape by 40.4%, 22.0%, and 23.9% on ABC, CADParser, and Automate, respectively, and improving zero-shot classification on FabWave by 15% in Top-1 score. We further demonstrate its utility as a CAD-aware similarity metric for evaluating text and image-conditioned CAD generation, establishing the importance of structure-aware pretraining for multimodal CAD understanding. Project page is available at https://muhammadusama100.github.io/BrepClip2026/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。