arXiv:2603.06617cs.LGcs.AI2026-03

Evo统一了自回归与扩散模型,让语言生成像进化一样渐进优化。

Evo: Autoregressive-Diffusion Large Language Models with Evolving Balance

  • 将文本生成看作潜在向量随时间演化的连续过程,动态平衡自回归与扩散策略。
  • 8B模型在15项任务上达到顶尖或接近顶尖水平,推理速度不降反而更快。
  • 适合追求高质量生成与高效推理的开发者,尤其看重逻辑推理能力者。

我们提出Evo,一种双模态潜在轨迹模型,将自回归(AR)与基于扩散的语言生成统一于连续演化生成框架中。不同于将二者视为独立范式,Evo将文本生成重构为潜在流:每个词元关联一个向量嵌入,其随进展变量 $t_i \in [0, 1]$ 演化,反映语义成熟度。低 $t_i$ 对应自信的AR式精炼,高 $t_i$ 则激发扩散式规划,使模型可根据不确定性自适应平衡。理论上,我们证明两者均是共享概率流的离散化形式,并从统一变分ELBO推导出Evo的训练目标。模型以时序条件Transformer实现,由共享向量场驱动,端到端训练以联合推断潜在码及其进展时间。解码时,Evo执行高效、语义感知的精炼,生成高质量输出且不牺牲速度。实证表明,Evo 8B在15个多样化基准测试中表现卓越,涵盖推理(GSM8K、ARC-C)、代码生成(HumanEval、MBPP)与通用语言理解,同时保持快速推理。结果表明,Evo为大模型设计带来新范式,兼具优异生成质量、鲁棒符号推理与解码效率。

原文摘要 · Abstract (English)

We introduce \textbf{Evo}, a duality latent trajectory model that bridges autoregressive (AR) and diffusion-based language generation within a continuous evolutionary generative framework. Rather than treating AR decoding and diffusion generation as separate paradigms, Evo reconceptualizes text generation as a latent flow: each token is associated with a vector-valued embedding that evolves over a progression variable $t_i \in [0, 1]$, indicating its semantic maturity. Low $t_i$ values correspond to confident AR-like refinement, while high values invoke diffusion-style planning, allowing the model to adaptively balance AR and diffusion based on uncertainty. Theoretically, we show that both AR and diffusion models emerge as discretizations of a shared probability flow, and we derive Evo's training objective from a unified variational ELBO. The model is implemented as a time-conditioned Transformer governed by a shared vector field, trained end-to-end to jointly infer latent codes and their progression times. During decoding, Evo performs efficient, semantics-aware refinement, achieving high-quality outputs without sacrificing speed. Empirically, Evo 8B achieves state-of-the-art or highly competitive results on 15 diverse benchmarks, including reasoning (GSM8K, ARC-C), code generation (HumanEval, MBPP), and general language understanding, while maintaining fast inference speed. Our results demonstrate that Evo delivers a new paradigm for LLM design with strong generation quality, robust symbolic reasoning, and decoding efficiency.

语言模型扩散模型生成效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。