用单图生成可交互3D动画,不依赖复杂建模
3DProxyImg: Controllable 3D-Aware Animation Synthesis from Single Image via 2D-3D Aligned Proxy Embedding
- 用粗略3D结构做骨架,图像生成模型负责细节
- 在低算力设备上实现高效动画生成
- 支持精确交互控制,适合内容创作者快速试错
3D动画是现代视觉媒体的核心,但传统制作流程仍高度依赖人力、专业技能且计算开销大。近期基于AIGC的方法虽部分自动化资产创建与绑定,却或延续完整3D管线的高成本,或依赖视频生成范式而牺牲3D可控性与交互能力。本文聚焦单图3D动画生成,指出进展受限于渲染质量与3D控制之间的权衡。为此,提出轻量级3D动画框架,将几何控制与外观合成解耦。核心是2D-3D对齐的代理表征:以粗略3D估计作为结构载体,将高保真外观与视图合成交由学习的图像空间生成先验完成。该代理形式使生成结果具备类传统管线的3D感知运动控制与交互能力,无需精确几何或昂贵优化,并自然扩展至连贯背景动画。大量实验表明,本方法可在低功耗平台高效生成动画,在身份保持、几何与纹理一致性以及用户可精确交互控制方面优于基于视频的3D动画生成方法。
原文摘要 · Abstract (English)
3D animation is central to modern visual media, yet traditional production pipelines remain labor-intensive, expertise-demanding, and computationally expensive. Recent AIGC-based approaches partially automate asset creation and rigging, but they either inherit the heavy costs of full 3D pipelines or rely on video-synthesis paradigms that sacrifice 3D controllability and interactivity. We focus on single-image 3D animation generation and argue that progress is fundamentally constrained by a trade-off between rendering quality and 3D control. To address this limitation, we propose a lightweight 3D animation framework that decouples geometric control from appearance synthesis. The core idea is a 2D-3D aligned proxy representation that uses a coarse 3D estimate as a structural carrier, while delegating high-fidelity appearance and view synthesis to learned image-space generative priors. This proxy formulation enables 3D-aware motion control and interaction comparable to classical pipelines, without requiring accurate geometry or expensive optimization, and naturally extends to coherent background animation. Extensive experiments demonstrate that our method achieves efficient animation generation on low-power platforms and outperforms video-based 3D animation generation in identity preservation, geometric and textural consistency, and the level of precise, interactive control it offers to users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。