arXiv:2606.22905cs.CV2026-06被引 5

实时生成连贯且懂用户意图的虚拟形象视频

InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars

论文配图:InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars
图 1 · 摘自论文原文
  • 用自回归蒸馏实现无限时长实时生成
  • 长短期视觉记忆保持形象一致性
  • 推理-反应模块支持复杂交互场景

基于扩散模型的最新进展已实现音频驱动的实时虚拟形象生成。然而,现有方法在复杂交互场景下难以维持视觉时序一致性,且无法显式感知用户意图。为此,我们提出InteractiveAvatar,一个支持视觉一致性和意图感知的实时无限流视频生成框架。通过自回归蒸馏,该框架可在任意长时长内实现人类形象的实时流式生成。为保障视觉一致性,我们引入长短期视觉记忆(LSVM)机制,将历史视觉信息灵活压缩为紧凑标记,同时保留短时相干性与长期一致性。为生成符合用户意图的语音与动作,我们设计了推理-反应模块(RRM),结合状态循环策略与缓存切换机制。在多样化场景下的大量实验表明,该方法在长时生成中达到当前最优的视觉一致性,并实现实时复杂人机交互。

原文摘要 · Abstract (English)

Recent diffusion-based models have enabled realistic audio-driven avatar generation in real-time streaming. However, existing approaches struggle to maintain visual temporal consistency and fail to explicitly perceive user intent in complex interactive streaming scenarios. To address these challenges, we propose InteractiveAvatar, a real-time infinite-streaming video generation framework that supports visually consistent avatar video generation and intent-aware interactions. With autoregressive distillation, InteractiveAvatar achieves real-time str-eaming generation of human avatars over arbitrarily long durations. For visual consistency, we introduce a Long-Short Visual Memory (LSVM) mechanism that flexibly compresses historical visual information into compact tokens, preserving both short-range coherence and long-term consistency. To generate avatars with speeches and actions aligned with user intent, we propose a Reasoning-Reaction Module (RRM), which incorporates a State-Cycling strategy and a Cache-Switching mechanism. Extensive experimental results over diverse scenarios demonstrate that our method achieves state-of-the-art visual consistency in long-duration generation, while enabling complex user-avatar interaction in real time.

虚拟形象实时生成交互感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。