Emu3.5用10万亿词训练,能原生跨模态预测世界状态。
Emu3.5: Native Multimodal Models are World Learners

- 用10万亿词的视频图文序列端到端预训练,统一预测下一帧图文。
- 推理加速20倍,生成长时序图像和复杂文本图效果媲美Gemini 2.5。
- 适合做多模态推理、开放世界交互与视频生成的研究者使用。
我们提出Emu3.5,一个大规模原生多模态世界模型,可原生预测视觉与语言的下一状态。Emu3.5在包含超过10万亿令牌的视觉-语言交错数据上进行端到端预训练,数据主要来自互联网视频的连续帧与字幕。模型天然支持交错输入输出,进一步通过大规模强化学习后训练以增强多模态推理与生成能力。为提升推理效率,我们提出离散扩散适配(DiDA),将逐标记解码转为双向并行预测,使单图像推理速度提升约20倍,性能无损。Emu3.5具备强原生多模态能力,包括长时序视觉-语言生成、任意内容到图像(X2I)生成及复杂文本丰富图像生成。其具备可泛化的世界建模能力,支持时空一致的世界探索与开放世界具身操作。在图像生成与编辑任务中表现接近Gemini 2.5 Flash Image(Nano Banana),并在一系列交错生成任务中表现更优。项目已开源:https://github.com/baaivision/Emu3.5。
原文摘要 · Abstract (English)
We introduce Emu3.5, a large-scale multimodal world model that natively predicts the next state across vision and language. Emu3.5 is pre-trained end-to-end with a unified next-token prediction objective on a corpus of vision-language interleaved data containing over 10 trillion tokens, primarily derived from sequential frames and transcripts of internet videos. The model naturally accepts interleaved vision-language inputs and generates interleaved vision-language outputs. Emu3.5 is further post-trained with large-scale reinforcement learning to enhance multimodal reasoning and generation. To improve inference efficiency, we propose Discrete Diffusion Adaptation (DiDA), which converts token-by-token decoding into bidirectional parallel prediction, accelerating per-image inference by about 20x without sacrificing performance. Emu3.5 exhibits strong native multimodal capabilities, including long-horizon vision-language generation, any-to-image (X2I) generation, and complex text-rich image generation. It also exhibits generalizable world-modeling abilities, enabling spatiotemporally consistent world exploration and open-world embodied manipulation across diverse scenarios and tasks. For comparison, Emu3.5 achieves performance comparable to Gemini 2.5 Flash Image (Nano Banana) on image generation and editing tasks and demonstrates superior results on a suite of interleaved generation tasks. We open-source Emu3.5 at https://github.com/baaivision/Emu3.5 to support community research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。