arXiv:2601.21237cs.DScs.CL2026-01被引 4

研究噪声对语言生成极限的影响,发现少量噪声即有决定性作用

Characterizing the Effect of Noise in Language Generation in the Limit

  • 通过形式化模型分析噪声串对生成能力的破坏
  • 单个噪声串就足以使可生成集合严格缩小
  • 单个噪声与任意有限噪声效果等价,颠覆此前层级认知

Kleinberg 和 Mullainathan 提出了一种研究语言生成现象的形式化框架——语言生成极限。在此模型中,对手提供未知目标语言的示例字符串序列,算法需在有限时间内正确生成未见过的目标字符串。Li、Raman 与 Tewari(2025)随后引入了非均匀与均匀生成的细化概念,Raman 与 Raman(2025)则提出允许对手插入额外字符串的噪声模型。本文研究该噪声模型下噪声的影响:首先证明,在均匀与非均匀生成中,单个噪声串会严格缩小可生成集合,回答了 Raman 与 Raman(2025)中的开放问题;其次证明,对于两类生成方式,单个噪声串的效果等同于任意有限数量噪声,与 Bai、Panigrahi 与 Zhang(2026)所揭示的严格层级形成鲜明对比。最后,基于前述结果,首次给出了非均匀噪声依赖可生成性的完整刻画。

原文摘要 · Abstract (English)

Kleinberg and Mullainathan recently proposed a formal framework for studying the phenomenon of language generation, called language generation in the limit. In this model, an adversary gives an enumeration of example strings from an unknown target language, and the algorithm is tasked with correctly generating unseen strings from the target language within finite time. Refined notions of non-uniform and uniform generation were later introduced by Li, Raman, and Tewari (2025), and a noisy model was introduced by Raman and Raman (2025), which allows the adversary to insert extraneous strings. A natural question in the noisy model is to quantify the effect of noise, by studying the impact of each additional extraneous string. We show two complementary results in this setting. We first show that for both uniform and non-uniform generation, a single noisy string strictly reduces the set of collections that can be generated, thus answering an open question in Raman and Raman (2025). Then, we show for both uniform and non-uniform generation that generation with a single noisy string is equivalent to generation with any finite amount of noise, sharply contrasting with the strict hierarchy for noisy generation in the limit shown by Bai, Panigrahi, and Zhang (2026). Finally, we leverage our previous results to provide the first known characterization for non-uniform noise-dependent generatability.

语言生成形式化分析噪声影响

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。