研究大模型用最短提示生成特定文本的复杂度,为提示工程提供量化标准。
Prompting Complexity: Shortest Prompts for Texts and Behaviors in LLMs

- 定义提示复杂度:最短合理提示使模型确定性输出目标文本
- 同一文本对不同模型可能便宜或昂贵,无通用不变性
- 适用于评估提示效率,适合提示优化与模型可访问性研究
本文提出提示复杂度的概念:在固定指令微调语言模型下,能使其确定解码生成目标文本的最短合理提示长度。该度量是资源有界柯尔莫哥洛夫复杂度的模型相关类比——提示为程序,模型接口为解释器,提示省略的信息由模型权重、训练分布、分词器、模板和解码规则提供。不同于经典复杂度,此度量故意非普适;在有限上下文设置中可通过枚举计算,但无模型无关不变性:相同文本对某些模型廉价,对另一些则不可达或昂贵。为保持搜索空间与提示工程一致,限制程序为合理可读文本而非任意标记串。将精确定义扩展至软提示复杂度以处理近似输出,形成模型相关文本压缩的有损概念,并为提示优化提供形式化目标。还定义了基于最短生成提示的提示距离,以及满足规范的任意输出行为的提示复杂度。基于这些框架,提出一个实证研究议程,探索在固定模型接口下,哪些文本和行为可通过短而合理的提示实现。
原文摘要 · Abstract (English)
In this paper, we define the quantity of prompting complexity: for a fixed instruction-tuned language model, what is the shortest plausible prompt that makes deterministic decoding produce a target text? It is an LM-relative analogue of resource-bounded Kolmogorov complexity: the prompt is a program, the model interface is the interpreter, and information omitted from the prompt is supplied by the model's weights, training distribution, tokenizer, template, and decoding rule. Unlike classical Kolmogorov complexity, this measure is intentionally non-universal. In the finite-context setting it is computable by enumeration, but there is no model-independent invariance theorem; the same text may be cheap for one model and inaccessible or expensive for another. To keep the search space aligned with prompt engineering, we restrict programs to plausible human-readable texts rather than arbitrary token strings. We extend the exact definition to soft prompting complexity for approximate outputs, yielding a lossy notion of model-relative text compression and a formal target for prompt optimization. We also define prompting distance by comparing shortest generating prompts, and behavioral prompting complexity for reaching any output satisfying a specification. Based on these formulations, we define a research agenda for empirically studying which texts and behaviors are accessible from short plausible prompts under a fixed LM interface.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。