arXiv:2606.26092cs.CV2026-06被引 2

让虚拟试穿视频可自由换视角,支持360度环视和子弹时间效果。

TryOnCrafter: Unleashing Camera Trajectories for Realistic Video Virtual Try-on via a Renderable 4D Try-on Proxy

论文配图:TryOnCrafter: Unleashing Camera Trajectories for Realistic Video Virtual Try-on via a Renderable 4D Try-on Proxy
图 1 · 摘自论文原文
  • 用可渲染的4D试穿代理分离人物与背景,实现结构与纹理解耦。
  • 支持任意相机轨迹运动,生成物理合理且视角自由的逼真视频。
  • 适用于重定位、子弹时间、360度观看等多样应用场景。

尽管视频虚拟试穿(VVT)在动态人体上合成逼真服装叠加方面取得显著进展,但现有方法仍受限于对源相机轨迹的被动依赖,难以实现全方位视角探索的交互自由。为此,我们提出全新的研究方向:相机可控视频虚拟试穿(CaM-VVT)。与传统VVT不同,CaM-VVT不仅要求视角无关的纹理生成,还需在任意非约束相机运动下,严格保持非刚性人体动态与背景环境的结构同步。为此,我们提出首个专为该任务设计的统一DiT框架——TryOnCrafter。不同于隐式像素空间操作,我们引入可渲染的4D试穿代理,显式解耦人体与环境。通过将高保真2D试穿先验提炼为基于3DGS的着装人体模型,并结合SMPL-X序列驱动动画,再与重建的背景点云进行度量对齐,构建出具有高纹理密度和运动完整性的稳健结构基础。其代理锚定视频DiT利用此结构作为几何主锚,确保生成的逼真视频严格遵循预设轨迹与物理合理形变。得益于4D代理的固有可编辑性,TryOnCrafter支持多种下游应用,包括人体重定位、'子弹时间'效果及360度轨道视角浏览。

原文摘要 · Abstract (English)

While Video Virtual Try-on (VVT) has achieved remarkable progress in synthesizing realistic garment overlays on dynamic subjects, existing paradigms remains fundamentally constrained by a passive dependency on source camera trajectories, failing to accommodate the requisite interactive freedom for omnidirectional viewpoint exploration. To address this limitation, we define a pioneering research frontier: Camera-controllable Video Virtual Try-on (CaM-VVT). Unlike conventional VVT, CaM-VVT not only necessitates viewpoint-agnostic texture hallucination but also strict structural synchronization between non-rigid human dynamics and background contexts under arbitrary, unconstrained camera movements. To tackle these challenges, we present TryOnCrafter, the first unified DiT-based framework specifically architected for the CaM-VVT task. Departing from implicit pixel-space manipulation, we introduce a Renderable 4D Try-on Proxy that explicitly decouples the human subject from the environment. This is achieved by distilling high-fidelity 2D try-on priors into a clothed 3DGS-based avatar, which is subsequently animated via SMPL-X sequences and metric-aligned into a reconstructed background point cloud. This proxy establishes a robust structural foundation with superior texture density and motion integrity. Our Proxy-Anchored Video DiT leverages this robust structural foundation as a primary geometric anchor, ensuring that the synthesized photorealistic videos are strictly constrained by prescribed trajectories and physically plausible deformations. Benefiting from the inherent editability of the 4D proxy, TryOnCrafter facilitates diverse downstream applications, including human relocalization, ``bullet time'' effects, and $360$-degree orbital viewing.

虚拟试穿360度视角4D建模视频生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。