用扩散模型生成形状间语义对齐的平滑插值，实现高质量点云注册。
SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimation
- 基于扩散模型生成多视角图像中间帧，驱动3D点云插值
- 通过动态高斯溅射重建中间点云，保持几何一致性
- 利用大视觉模型提升语义关联，支持语义合理的形变
本文提出SRIF，一种基于扩散图像形变与流估计的语义形状注册框架。给定一对外参对齐的形状，先从多视角渲染图像，再利用基于扩散模型的图像插值框架生成两者间的中间图像序列。这些图像被输入动态3D高斯溅射框架，重建并后处理得到符合图像形变过程的中间点云。最后，针对上述流程设计了一种新型注册模块，估计连续归一化流，将源形状一致地变形至目标形状，以中间点云作为弱引导。核心思想是借助大视觉模型（LVMs）建立形状间的语义关联，从而获得比传统特征提取与对齐更丰富的形状关系信息。实验表明，SRIF在挑战性形状对上实现了高质量稠密对应，并在中间过程提供平滑且语义合理的插值。代码已开源。
原文摘要 · Abstract (English)
In this paper, we propose SRIF, a novel Semantic shape Registration framework based on diffusion-based Image morphing and Flow estimation. More concretely, given a pair of extrinsically aligned shapes, we first render them from multi-views, and then utilize an image interpolation framework based on diffusion models to generate sequences of intermediate images between them. The images are later fed into a dynamic 3D Gaussian splatting framework, with which we reconstruct and post-process for intermediate point clouds respecting the image morphing processing. In the end, tailored for the above, we propose a novel registration module to estimate continuous normalizing flow, which deforms source shape consistently towards the target, with intermediate point clouds as weak guidance. Our key insight is to leverage large vision models (LVMs) to associate shapes and therefore obtain much richer semantic information on the relationship between shapes than the ad-hoc feature extraction and alignment. As a consequence, SRIF achieves high-quality dense correspondences on challenging shape pairs, but also delivers smooth, semantically meaningful interpolation in between. Empirical evidence justifies the effectiveness and superiority of our method as well as specific design choices. The code is released at https://github.com/rqhuang88/SRIF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。