研究大模型能生成哪些概率分布，发现高低熵分布更容易实现。
Distribution Prompting: Understanding the Expressivity of Language Models Through the Next-Token Distributions They Can Produce
- 用梯度优化提示词，让模型输出指定的下一个词概率分布。
- 低熵或高熵分布比中等熵分布更容易诱导生成。
- 模型自动生成的分布比随机分布更易复现，适合做概率建模参考。
自回归语言模型在给定提示词时，会为每个时间步生成一个词的概率分布。本文系统性地研究了语言模型能够产生的概率分布，发现某些分布比其他分布更难诱导。具体而言，对于词汇表上的任意目标下一个词分布，我们通过软或硬的梯度提示调优方法，尝试找到能使其输出分布尽可能接近目标的提示词。结果表明：（1）总体上，熵值极低或极高的分布比中等熵的分布更容易近似；（2）在相同熵的情况下，包含“异常词”的分布更容易近似；（3）由语言模型自身生成的目标分布——即使使用不同分词器——也比随机选择的目标分布更容易近似。这些发现揭示了语言模型的表达能力及其作为概率分布提议者的局限性。
原文摘要 · Abstract (English)
Autoregressive neural language models (LMs) generate a probability distribution over tokens at each time step given a prompt. In this work, we attempt to systematically understand the probability distributions that LMs can produce, showing that some distributions are significantly harder to elicit than others. Specifically, for any target next-token distribution over the vocabulary, we attempt to find a prompt that induces the LM to output a distribution as close as possible to the target, using either soft or hard gradient-based prompt tuning. We find that (1) in general, distributions with very low or very high entropy are easier to approximate than those with moderate entropy; (2) among distributions with the same entropy, those containing ''outlier tokens'' are easier to approximate; (3) target distributions generated by LMs -- even LMs with different tokenizers -- are easier to approximate than randomly chosen targets. These results offer insights into the expressiveness of LMs and the challenges of using them as probability distribution proposers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。