用预训练模型直接编辑3D网格，仅凭单图就能精准修改形状和纹理。
VecSet-Edit: Unleashing Pre-trained LRM for Mesh Editing from Single Image
- 基于向量集令牌的空间特性，用2D图像定位网格区域。
- 提出去噪时剔除几何异常的漂移感知修剪策略，提升精度。
- 适合需要高保真3D编辑的设计师或工业建模用户。
3D编辑已成为灵活操控3D资产的关键研究方向。尽管现有方法多聚焦于3D高斯溅射或多视角图像，对3D网格的直接编辑仍研究不足。以往工作如VoxHammer依赖体素表示，存在分辨率有限且需人工制作3D掩码的问题。为此，我们提出首个利用高保真VecSet大重建模型（LRM）作为主干的网格编辑框架——VecSet-Edit。通过对VecSet令牌空间特性的分析，发现令牌子集控制不同几何区域。基于此，我们设计了掩码引导的令牌播种与注意力对齐的令牌门控策略，仅通过2D图像条件精确定位目标区域。同时，针对VecSet扩散过程与体素的差异，提出漂移感知令牌修剪，在去噪过程中剔除几何异常点。最后，细节保持的纹理烘焙模块确保原网格的几何与纹理信息均被保留。更多细节见项目页：https://github.com/BlueDyee/VecSet-Edit/tree/main
原文摘要 · Abstract (English)
3D editing has emerged as a critical research area to provide users with flexible control over 3D assets. While current editing approaches predominantly focus on 3D Gaussian Splatting or multi-view images, the direct editing of 3D meshes remains underexplored. Prior attempts, such as VoxHammer, rely on voxel-based representations that suffer from limited resolution and necessitate labor-intensive 3D mask. To address these limitations, we propose \textbf{VecSet-Edit}, the first pipeline that leverages the high-fidelity VecSet Large Reconstruction Model (LRM) as a backbone for mesh editing. Our approach is grounded on a analysis of the spatial properties in VecSet tokens, revealing that token subsets govern distinct geometric regions. Based on this insight, we introduce Mask-guided Token Seeding and Attention-aligned Token Gating strategies to precisely localize target regions using only 2D image conditions. Also, considering the difference between VecSet diffusion process versus voxel we design a Drift-aware Token Pruning to reject geometric outliers during the denoising process. Finally, our Detail-preserving Texture Baking module ensures that we not only preserve the geometric details of original mesh but also the textural information. More details can be found in our project page: https://github.com/BlueDyee/VecSet-Edit/tree/main
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。