通过细粒度强化学习,让3D网格生成更精准、结构更合理。
Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuning
- 用面级质量掩码与偏好优化,实现逐面精修。
- 相比预训练模型,网格误差降低24.6%,拓扑得分提升3.8%。
- 适合需要高质量3D网格的工业级建模与生成任务。
现有3D网格生成的预训练模型常受数据偏差影响,生成质量较低;而全局强化学习方法依赖物体级奖励,难以捕捉局部结构细节。为此,我们提出Mesh-RFT,一种基于掩码直接偏好优化(M-DPO)的细粒度强化微调框架,通过质量感知的面级掩码实现局部精修。为高效评估质量,引入拓扑感知评分系统,从物体和面两个层面分别以边界边比例(BER)和拓扑得分(TS)衡量几何完整性和拓扑规则性。将这些指标融入细粒度强化学习策略,使Mesh-RFT成为首个在单个面粒度上优化网格质量的方法,在修复局部错误的同时保持全局一致性。实验表明,该方法相比预训练模型将霍夫德距离(HD)降低24.6%,拓扑得分(TS)提升3.8%;相比全局DPO方法,HD降低17.4%,TS提升4.9%。结果证明Mesh-RFT能有效提升几何完整性与拓扑规则性,达到生产级网格生成的新最优水平。
原文摘要 · Abstract (English)
Existing pretrained models for 3D mesh generation often suffer from data biases and produce low-quality results, while global reinforcement learning (RL) methods rely on object-level rewards that struggle to capture local structure details. To address these challenges, we present Mesh-RFT, a novel fine-grained reinforcement fine-tuning framework that employs Masked Direct Preference Optimization (M-DPO) to enable localized refinement via quality-aware face masking. To facilitate efficient quality evaluation, we introduce an objective topology-aware scoring system to evaluate geometric integrity and topological regularity at both object and face levels through two metrics: Boundary Edge Ratio (BER) and Topology Score (TS). By integrating these metrics into a fine-grained RL strategy, Mesh-RFT becomes the first method to optimize mesh quality at the granularity of individual faces, resolving localized errors while preserving global coherence. Experiment results show that our M-DPO approach reduces Hausdorff Distance (HD) by 24.6% and improves Topology Score (TS) by 3.8% over pre-trained models, while outperforming global DPO methods with a 17.4% HD reduction and 4.9% TS gain. These results demonstrate Mesh-RFT's ability to improve geometric integrity and topological regularity, achieving new state-of-the-art performance in production-ready mesh generation. Project Page: https://hitcslj.github.io/mesh-rft/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。