UniVid统一视频建模，生成理解兼备且无需重训
UniVid: The Open-Source Unified Video Model
- 用轻量适配器连接多模态大模型与扩散解码器
- 在长视频评测中提升2.2%，问答任务准确率增1.0%-3.3%
- 适合需要高效视频生成与理解的开发者和研究者
统一视频建模结合生成与理解能力日益重要，但面临两大挑战：基于流的生成中因文本-视觉令牌失衡导致语义失真，以及跨模态注意力在时序轨迹上表现受限；同时，将图像中心的多模态大模型（MLLM）扩展至视频需高昂重训练成本。我们提出UniVid，通过轻量适配器将MLLM与扩散解码器耦合，实现视频生成与理解。引入温度模态对齐以增强提示遵循性，提出金字塔反射机制通过动态关键帧选择实现高效时序推理。在标准基准测试中表现卓越，相较EasyAnimateV5.1在VBench-Long总分提升2.2%，在MSVD-QA与ActivityNet-QA上分别领先最佳7B基线1.0%和3.3%。代码与官网详见：https://github.com/AIGeeksGroup/UniVid, https://aigeeksgroup.github.io/UniVid。
原文摘要 · Abstract (English)
Unified video modeling that combines generation and understanding capabilities is increasingly important but faces two key challenges: maintaining semantic faithfulness during flow-based generation due to text-visual token imbalance and the limitations of uniform cross-modal attention across the flow trajectory, and efficiently extending image-centric MLLMs to video without costly retraining. We present UniVid, a unified architecture that couples an MLLM with a diffusion decoder through a lightweight adapter, enabling both video understanding and generation. We introduce Temperature Modality Alignment to improve prompt adherence and Pyramid Reflection for efficient temporal reasoning via dynamic keyframe selection. Extensive experiments on standard benchmarks demonstrate state-of-the-art performance, achieving a 2.2% improvement on VBench-Long total score compared to EasyAnimateV5.1, and 1.0% and 3.3% accuracy gains on MSVD-QA and ActivityNet-QA, respectively, compared with the best prior 7B baselines. Code: https://github.com/AIGeeksGroup/UniVid. Website: https://aigeeksgroup.github.io/UniVid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。