arXiv:2606.08674cs.CVcs.AI2026-06

让生物行为视频自适应生成时长,更真实。

BioVid: Autoregressive Video Generation with Biological Behavior Semantic Comprehension

  • 用2D编码+3D解码架构,让视频时长由模型学习决定。
  • 在真实动作时长分布上误差仅1.24帧,远优于固定长度模型。
  • 适合需要精准时长模拟的生物行为研究与动画生成。

生物行为视频生成不仅需要视觉合理,动作时长本身也是语义属性。现有模型多依赖固定时长窗口、外部续接或提示驱动故事,导致时长由外部指定而非从行为中学习。为此,我们提出BioVid,一种数据驱动的自回归框架,可生成自适应长度的生物行为视频。BioVid采用2D-encode/3D-decode分词器:二维FSQ-R3GAN编码器将每帧转为离散视觉标记,保留单帧信息以支持下一帧预测和基于EOS的终止;三维解码器通过时间膨胀与视频微调,重建标记序列并减少闪烁。因果Transformer建模帧级标记序列,仅以首帧为条件,在发出序列结束标记(EOS)时停止生成,使时长自然源于行为分布。我们在NTU RGB+D数据集的A001饮水动作上评估,对94个未见片段,BioVid与真实时长分布的Wasserstein-1距离为1.24帧。相较之下,固定长度基线即使调整至最接近数据集均值的长度,误差仍约6-7帧;使用常规16帧生成长度时误差达约15帧。结果表明,BioVid能有效学习并复现生物行为的内在时长分布。

原文摘要 · Abstract (English)

Video generation for biological behavior requires more than visually plausible motion: the duration of an action is itself a semantic property. Existing models usually rely on fixed temporal windows, external continuation, or prompt-driven stories, so length is specified externally rather than learned from behavior. To address this gap, we propose BioVid, a data-driven autoregressive framework for adaptive-length biological behavior generation. BioVid uses a 2D-encode/3D-decode tokenizer: a two-dimensional FSQ-R3GAN encoder converts each frame into discrete visual tokens, preserving single-frame information suited for next-token prediction and EOS-based termination, while a temporally inflated and video-finetuned three-dimensional decoder reconstructs generated tokens with temporal context to reduce flickering. A causal Transformer then models the frame-wise token sequence and, conditioned only on the first frame, stops generation when it emits an End-of-Sequence token, allowing duration to emerge from the learned behavior distribution. We evaluate BioVid on the A001 drinking action from NTU RGB+D. On 94 held-out clips, BioVid achieves a Wasserstein-1 distance of 1.24 frames from the real duration distribution. In comparison, fixed-length baselines yield distances of approximately 6-7 frames even when configured to the available length closest to the dataset mean, and approximately 15 frames when using the conventional 16-frame generation length. These results demonstrate the ability of BioVid to learn and reproduce the intrinsic duration distribution of biological behavior.

视频生成自回归生物行为

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。