arXiv:2512.10054cs.AIcs.CL2025-12

让大模型同时生成多个内容分支,像并行流水线一样高效写作。

Parallel Decoder Transformer: Planner-Conditioned Latent Coordination for Model-Intrinsic Parallel Generation

  • 用三个独立解码器并行处理不同内容规划,共享底层知识但各自独立生成
  • 通过持续规划记忆和延迟消息传递实现跨分支协调,保持每轮生成的条件独立性
  • 适合需要多线程内容创作的研究者,尤其关注可验证的生成一致性与可控性

自回归语言模型一次仅生成一个词,即使文档中存在可并行撰写的章节。现有并行生成系统依赖外部结构改造模型。本文提出模型内生并行生成:单一训练架构拥有多个因果前沿,在每轮同步解码中为每个前沿输出一个下一个词的概率分布。平行解码器变压器(PDT)保留冻结的共享底层知识主干,用三个独立参数化的物理解码堆栈替换上层结构。提示阶段的规划器生成三个无序连续大纲,分别硬路由至一个解码器作为持久化计划-键值记忆;有限乘积量化笔记总线在解码器间传递块延迟的潜在信息。每条路径内部仍保持自回归,同轮次各路径输出在源文本、计划、私有历史及先前已提交消息条件下条件独立。我们设计了面向长篇历史叙述的源基监督机制,包含单主人引用事实与词级跨路径依赖,复合目标函数、分阶段课程训练,以及预注册的因果评估:计划互换与移除、延迟消息消融、参数匹配的纯自回归对照、依赖词似然度测试与盲测人类事实审计。架构与评估流程已实现;科学训练与保留集评估正在进行中。本文呈现理论、设计与可证伪协议,不报告正向实证结果。

原文摘要 · Abstract (English)

Autoregressive language models expose one causal token frontier, even when the requested document contains sections that could be developed concurrently. Existing parallel-generation systems arrange external branches around an otherwise unchanged model. We instead formulate model-intrinsic parallel generation: a single trained architecture owns multiple causal frontiers and produces one next-token distribution for each frontier in every synchronized decoding round. The Parallel Decoder Transformer (PDT) retains a frozen shared lower knowledge trunk and replaces the upper trunk with three independently parameterized physical decoder stacks. A prompt-time set planner produces three unordered continuous outlines, each hard-routed to one decoder as persistent Plan-KV memory, while a finite product-quantized notes bus carries block-delayed latent messages among the decoders. Autoregression is preserved within each lane; same-round lane tokens are conditionally independent given the source, plans, private histories, and previously committed messages. We specify source-grounded supervision for long-form historical exposition with single-owner cited facts and token-aligned cross-lane dependencies, a composite objective, a staged curriculum, and preregistered causal evaluations: plan swap and removal, delayed-message ablation, a parameter-matched self-only control, dependency-token likelihood, and blinded human fact audits. The architecture and evaluation pipeline are implemented; scientific training and held-out evaluation are in progress. This paper presents the theory, design, and falsifiable protocol, not a positive empirical result.

并行生成解码器设计多分支协同可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。