综述交互式生成视频技术,涵盖游戏、智能体训练与自动驾驶三大应用。
A Survey of Interactive Generative Video
- 提出五模块框架:生成、控制、记忆、动态、智能,系统化拆解交互视频核心组件。
- 总结三类典型应用:虚拟世界探索、多模态智能体训练、自动驾驶闭环仿真测试。
- 指出实时生成、开放域控制、长期一致性等关键挑战,适合研究者参考方向。
交互式生成视频(IGV)作为应对各领域对高质量互动视频内容日益增长需求的关键技术应运而生。本文将IGV定义为融合生成能力与交互功能的技术,可通过控制信号实现用户参与,并提供响应式反馈。我们系统综述了IGV在三大主要领域的应用:1)游戏领域,支持虚拟世界的无限探索;2)具身智能(embodied AI),作为物理感知环境生成器,用于训练智能体在动态演化场景中进行多模态交互;3)自动驾驶,提供闭环仿真能力,用于安全关键测试与验证。为推动未来发展,我们提出一个包含五个核心模块的理想IGV系统框架:生成、控制、记忆、动态与智能。进一步分析了各模块实现中的技术挑战与未来方向,如实现实时生成、支持开放域控制、保持长期连贯性、模拟精准物理过程以及集成因果推理。我们认为该系统性分析将促进IGV领域的研究与开发,助力技术向更复杂实用的应用演进。
原文摘要 · Abstract (English)
Interactive Generative Video (IGV) has emerged as a crucial technology in response to the growing demand for high-quality, interactive video content across various domains. In this paper, we define IGV as a technology that combines generative capabilities to produce diverse high-quality video content with interactive features that enable user engagement through control signals and responsive feedback. We survey the current landscape of IGV applications, focusing on three major domains: 1) gaming, where IGV enables infinite exploration in virtual worlds; 2) embodied AI, where IGV serves as a physics-aware environment synthesizer for training agents in multimodal interaction with dynamically evolving scenes; and 3) autonomous driving, where IGV provides closed-loop simulation capabilities for safety-critical testing and validation. To guide future development, we propose a comprehensive framework that decomposes an ideal IGV system into five essential modules: Generation, Control, Memory, Dynamics, and Intelligence. Furthermore, we systematically analyze the technical challenges and future directions in realizing each component for an ideal IGV system, such as achieving real-time generation, enabling open-domain control, maintaining long-term coherence, simulating accurate physics, and integrating causal reasoning. We believe that this systematic analysis will facilitate future research and development in the field of IGV, ultimately advancing the technology toward more sophisticated and practical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。