arXiv:2602.10796cs.LG2026-02

PRISM让序列生成模型既快又强,突破了效率与表达力的瓶颈。

PRISM: Parallel Residual Iterative Sequence Model

  • 通过并行化迭代更新机制,重构了TTT的残差迭代模式。
  • 在保持性能的同时,吞吐量提升174倍,远超传统方法。
  • 适合追求高效生成、需高吞吐场景的开发者使用。

生成式序列建模面临Transformer表达力与线性模型效率之间的根本矛盾。现有高效架构受制于浅层单步线性更新的理论限制,而强大的迭代方法如测试时训练(TTT)因存在令牌级状态依赖与步骤级循环依赖,破坏硬件并行性。本文提出PRISM(并行残差迭代序列模型),以并行方式重构TTT中的表达性残差迭代结构。采用写-遗忘解耦策略,将非线性注入操作隔离。为规避显式求解器的串行依赖,PRISM设计两阶段代理架构:短卷积利用局部历史能量锚定初始残差,学习预测器则直接从输入估计修正更新。该设计将迭代修正的结构特征提炼为可并行前馈算子。理论上,证明该形式实现秩-L累积,结构上扩展了更新流形,突破单步秩-1瓶颈。实验表明,其性能媲美显式优化方法,同时实现174倍更高的吞吐量。代码已开源。

原文摘要 · Abstract (English)

Generative sequence modeling faces a fundamental tension between the expressivity of Transformers and the efficiency of linear sequence models. Existing efficient architectures are theoretically bounded by shallow, single-step linear updates, while powerful iterative methods like Test-Time Training (TTT) break hardware parallelism due to two dimensions of serial dependency: token-level state reliance and step-level iteration loops. We propose PRISM (Parallel Residual Iterative Sequence Model) to resolve this tension. PRISM explicitly reconstructs the expressive gate x residual x direction iteration pattern of TTT in a parallelizable form. We employ a Write-Forget Decoupling strategy that isolates non-linearity within the injection operator. To bypass the serial dependency of explicit solvers, PRISM utilizes a two-stage proxy architecture: a short-convolution anchors the initial residual using local history energy, while a learned predictor estimates the refinement updates directly from the input. This design distills structural patterns associated with iterative correction into a parallelizable feedforward operator. Theoretically, we prove that this formulation achieves Rank-$L$ accumulation, structurally expanding the update manifold beyond the single-step Rank-$1$ bottleneck. Empirically, it achieves comparable performance to explicit optimization methods while achieving \textbf{174x higher throughput}. Codes are available in https://github.com/gpr-prism/prism/.

序列建模并行计算生成模型效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。