用扩散模型生成3D形状描述，无需复杂表示或额外分类器。
Diff-3DCap: Shape Captioning with Diffusion Models
- 通过投影视图+连续扩散模型实现3D形状描述
- 性能媲美当前最先进方法，无需昂贵体素表示
- 利用预训练视觉语言模型嵌入作为引导信号
3D形状描述任务在计算机图形学领域占据重要地位，近年来受到广泛关注。传统方法常依赖昂贵的体素表示或目标检测技术，但效果往往不理想。为此，本文提出Diff-3DCap，采用一系列投影视图表示3D物体，并利用连续扩散模型完成描述生成。具体而言，在前向过程通过高斯噪声扰动嵌入式描述，在反向过程预测重建标注。扩散框架中嵌入预训练视觉-语言模型获取的视觉嵌入，自然充当引导信号，无需额外分类器。大量实验表明，Diff-3DCap性能可与当前最先进方法相媲美。
原文摘要 · Abstract (English)
The task of 3D shape captioning occupies a significant place within the domain of computer graphics and has garnered considerable interest in recent years. Traditional approaches to this challenge frequently depend on the utilization of costly voxel representations or object detection techniques, yet often fail to deliver satisfactory outcomes. To address the above challenges, in this paper, we introduce Diff-3DCap, which employs a sequence of projected views to represent a 3D object and a continuous diffusion model to facilitate the captioning process. More precisely, our approach utilizes the continuous diffusion model to perturb the embedded captions during the forward phase by introducing Gaussian noise and then predicts the reconstructed annotation during the reverse phase. Embedded within the diffusion framework is a commitment to leveraging a visual embedding obtained from a pre-trained visual-language model, which naturally allows the embedding to serve as a guiding signal, eliminating the need for an additional classifier. Extensive results of our experiments indicate that Diff-3DCap can achieve performance comparable to that of the current state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。