用可微奖励直接优化3D纹理生成，让结果更贴合几何结构和用户偏好。
End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards
- 将人类反馈转为可微奖励，端到端优化3D纹理生成过程。
- 在多个指标上优于当前最佳方法，用户偏好测试中显著胜出。
- 适合需要精准控制纹理外观与三维结构一致性的应用者。
尽管近期的3D生成模型能产出高质量纹理图像，但往往无法捕捉人类偏好或满足特定任务需求。3D纹理生成的核心挑战在于，多数方法依赖反复调用2D文生图模型，缺乏对输入3D网格对象三维结构的内在理解。为此，我们提出一种无需强化学习、端到端可微的框架，将人类反馈以可微奖励函数的形式直接嵌入3D纹理合成流程。通过在几何与外观模块间反向传播偏好信号，该方法生成的纹理不仅尊重3D几何结构，还符合预期标准。我们引入三种新颖的几何感知奖励函数，为从自然语言生成高质量3D内容提供更可控、可解释的路径。通过定性、定量及用户偏好评估，证明本方法在多个方面持续优于现有主流方法。实现代码已公开于：https://github.com/AHHHZ975/Differentiable-Texture-Learning
原文摘要 · Abstract (English)
While recent 3D generative models can produce high-quality texture images, they often fail to capture human preferences or meet task-specific requirements. Moreover, a core challenge in the 3D texture generation domain is that most existing approaches rely on repeated calls to 2D text-to-image generative models, which lack an inherent understanding of the 3D structure of the input 3D mesh object. To alleviate these issues, we propose an end-to-end differentiable, reinforcement-learning-free framework that embeds human feedback, expressed as differentiable reward functions, directly into the 3D texture synthesis pipeline. By back-propagating preference signals through both geometric and appearance modules of the proposed framework, our method generates textures that respect the 3D geometry structure and align with desired criteria. To demonstrate its versatility, we introduce three novel geometry-aware reward functions, which offer a more controllable and interpretable pathway for creating high-quality 3D content from natural language. By conducting qualitative, quantitative, and user-preference evaluations against state-of-the-art methods, we demonstrate that our proposed strategy consistently outperforms existing approaches. Our implementation code is publicly available at: https://github.com/AHHHZ975/Differentiable-Texture-Learning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。