仅用一段视频就能重建并预测3D流体，突破了以往需多视角视频的限制。
FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Video
- 通过生成多视角视频作为参考，实现单视频3D流体重建
- 可同时完成动态新视角生成、未来状态预测和交互模拟
- 适合做流体模拟、影视特效与机器人视觉的研究者
我们研究从单段视频中重建并预测3D流体外观与速度。现有方法需要多视角视频才能完成流体重建。本文提出FluidNexus框架，将视频生成与物理仿真相结合，以解决该问题。核心思想是合成多个新视角视频作为重建参考。FluidNexus包含两个关键组件：(1) 新视角视频合成器，结合逐帧视角生成与视频扩散优化，生成逼真视频；(2) 物理融合粒子表示，整合可微分仿真与渲染，实现3D流体重建与预测的同步。为评估方法，我们收集了两个新真实世界流体数据集，包含纹理背景和物体交互。所提方法支持从单个流体视频实现动态新视角生成、未来预测及交互模拟。项目网站：https://yuegao.me/FluidNexus。
原文摘要 · Abstract (English)
We study reconstructing and predicting 3D fluid appearance and velocity from a single video. Current methods require multi-view videos for fluid reconstruction. We present FluidNexus, a novel framework that bridges video generation and physics simulation to tackle this task. Our key insight is to synthesize multiple novel-view videos as references for reconstruction. FluidNexus consists of two key components: (1) a novel-view video synthesizer that combines frame-wise view synthesis with video diffusion refinement for generating realistic videos, and (2) a physics-integrated particle representation coupling differentiable simulation and rendering to simultaneously facilitate 3D fluid reconstruction and prediction. To evaluate our approach, we collect two new real-world fluid datasets featuring textured backgrounds and object interactions. Our method enables dynamic novel view synthesis, future prediction, and interaction simulation from a single fluid video. Project website: https://yuegao.me/FluidNexus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。