用强化学习让3D网格生成更精细准确，兼顾美观与结构完整
DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning
- 采用新型分词算法和预训练策略提升网格生成效率
- 通过人类偏好优化使生成网格在精度与视觉质量上超越现有方法
- 适合需要高质量3D建模的工业设计与游戏开发场景
三角网格在3D应用中对高效操作与渲染至关重要。尽管自回归方法通过预测离散顶点标记生成结构化网格，但常受限于面数不足与网格不完整问题。为此，我们提出DeepMesh框架，包含两项关键创新：(1) 一种高效的预训练策略，结合新型分词算法及数据清洗与处理优化；(2) 将强化学习引入3D网格生成，通过直接偏好优化（DPO）实现与人类偏好的对齐。我们设计了一套融合人工评估与3D指标的评分标准，用于收集偏好对以支持DPO，确保生成结果兼具视觉美感与几何准确性。在点云和图像条件下，DeepMesh可生成具有精细细节和精确拓扑的网格，在精度与质量上均优于当前最先进方法。
原文摘要 · Abstract (English)
Triangle meshes play a crucial role in 3D applications for efficient manipulation and rendering. While auto-regressive methods generate structured meshes by predicting discrete vertex tokens, they are often constrained by limited face counts and mesh incompleteness. To address these challenges, we propose DeepMesh, a framework that optimizes mesh generation through two key innovations: (1) an efficient pre-training strategy incorporating a novel tokenization algorithm, along with improvements in data curation and processing, and (2) the introduction of Reinforcement Learning (RL) into 3D mesh generation to achieve human preference alignment via Direct Preference Optimization (DPO). We design a scoring standard that combines human evaluation with 3D metrics to collect preference pairs for DPO, ensuring both visual appeal and geometric accuracy. Conditioned on point clouds and images, DeepMesh generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality. Project page: https://zhaorw02.github.io/DeepMesh/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。