用Transformer快速生成动态3D资产，支持连续时间插值与纹理追踪。
LIM: Large Interpolator Model for Dynamic Reconstruction
- 基于因果一致性损失的Transformer模型，直接前向推理插值3D隐式表示。
- 在1秒内完成任意时间点的高质量动态3D重建，支持显式网格序列输出。
- 适用于多类别动态资产重建，适合影视制作与实时渲染场景。
从视频数据中重建动态资产是计算机视觉与图形学中的核心任务。现有4D重建方法受限于特定类别模型或依赖慢速优化过程。受大型重建模型（LRM）启发，本文提出大型插值模型（LIM），一种基于Transformer的前馈式解决方案，通过新颖的因果一致性损失，实现时间维度上隐式3D表示的插值。给定时间 $t_0$ 与 $t_1$ 的隐式3D表示，LIM可在任意连续时间 $t\in[t_0,t_1]$ 生成形变后的三维形状，秒级生成高质量插值帧。此外，LIM支持显式网格跨时间追踪，输出一致的UV贴图网格序列，可直接集成至现有生产管线。我们还将LIM与基于扩散的多视角生成器结合，实现单目视频到动态4D重建。在多个动态数据集上评估，相较图像空间插值方法（如FiLM）和直接三平面线性插值，展现显著优势。总之，LIM是首个能在多种类别上实现高速、可追踪4D资产重建的前馈模型。
原文摘要 · Abstract (English)
Reconstructing dynamic assets from video data is central to many in computer vision and graphics tasks. Existing 4D reconstruction approaches are limited by category-specific models or slow optimization-based methods. Inspired by the recent Large Reconstruction Model (LRM), we present the Large Interpolation Model (LIM), a transformer-based feed-forward solution, guided by a novel causal consistency loss, for interpolating implicit 3D representations across time. Given implicit 3D representations at times $t_0$ and $t_1$, LIM produces a deformed shape at any continuous time $t\in[t_0,t_1]$, delivering high-quality interpolated frames in seconds. Furthermore, LIM allows explicit mesh tracking across time, producing a consistently uv-textured mesh sequence ready for integration into existing production pipelines. We also use LIM, in conjunction with a diffusion-based multiview generator, to produce dynamic 4D reconstructions from monocular videos. We evaluate LIM on various dynamic datasets, benchmarking against image-space interpolation methods (e.g., FiLM) and direct triplane linear interpolation, and demonstrate clear advantages. In summary, LIM is the first feed-forward model capable of high-speed tracked 4D asset reconstruction across diverse categories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。