用编辑操作构建序列生成流，实现灵活非自回归生成。
Edit Flows: Flow Matching with Edit Operations
- 通过插入、删除、替换操作定义序列上的连续时间马尔可夫过程。
- 在图像描述任务上超越自回归与掩码模型，在文本和代码生成中显著优于掩码构造。
- 适合需要灵活序列生成的场景，如自然语言和代码生成。
自回归生成模型天然支持变长序列生成，而非自回归模型往往受限于固定的词元结构。本文提出 Edit Flows，一种通过编辑操作（插入、删除、替换）在序列空间上定义离散流的非自回归模型。通过将这些操作建模为序列空间上的连续时间马尔可夫链，该方法实现了更贴近序列数据结构的灵活、位置相关生成。训练采用扩展状态空间与辅助变量，使学习过程高效且可计算。实验表明，Edit Flows 在图像描述任务上优于自回归与掩码模型，并在文本和代码生成中显著超越掩码构造方法。
原文摘要 · Abstract (English)
Autoregressive generative models naturally generate variable-length sequences, while non-autoregressive models struggle, often imposing rigid, token-wise structures. We propose Edit Flows, a non-autoregressive model that overcomes these limitations by defining a discrete flow over sequences through edit operations$\unicode{x2013}$insertions, deletions, and substitutions. By modeling these operations within a Continuous-time Markov Chain over the sequence space, Edit Flows enable flexible, position-relative generation that aligns more closely with the structure of sequence data. Our training method leverages an expanded state space with auxiliary variables, making the learning process efficient and tractable. Empirical results show that Edit Flows outperforms both autoregressive and mask models on image captioning and significantly outperforms the mask construction in text and code generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。