用一个学习到的标记替代长系统提示，大幅节省推理成本。
Learning a Single Token to Replace Long System Prompts in LLMs
- 训练一个单一令牌[BE]，通过重建和行为蒸馏保留原提示语义与效果。
- 在三个数据集上实现3000倍压缩比，下游性能保留约98%。
- 无需修改模型权重或额外模型，适合资源受限场景部署。
长系统提示被广泛用于引导大语言模型，但每次推理时重复处理会效率低下并占用宝贵的上下文空间。这引发核心问题：能否仅用一个极简的可学习表示来保留长提示的行为效果？为此，我们提出一种轻量级训练框架，学习一个行为等价令牌[BE]。该框架首先通过重建任务让[BE]编码原始系统提示的语义内容，再将提示的下游行为蒸馏至该单一令牌中。重要的是，该方法无需更新预训练模型权重、不依赖辅助压缩模型，也无需标注回复。在三个数据集上的实证评估表明，用单个[BE]令牌替换长提示可实现高达3000倍的提示压缩比，同时保持原提示约98%的下游性能。这显著降低了推理开销，并几乎释放整个上下文窗口用于用户输入和模型输出。
原文摘要 · Abstract (English)
Long system prompts are widely used to steer Large Language Models (LLMs), but repeatedly processing them at inference time is inefficient and consumes valuable context budget. This motivates a central question: can the behavioral effect of a long system prompt be retained using only a minimal learned representation? To enable this, we propose a lightweight training framework that learns a single Behavior-Equivalent Token ([BE]). The framework first trains [BE] to encode the semantic content of the original system prompt via reconstruction, and then distills the prompt's downstream behavior into this single token. Importantly, our method requires no update to the pretrained LLM weights, no auxiliary compression models, and no labeled responses. Empirical evaluations on three datasets show that replacing long prompts with a single [BE] token yields up to a $3000\times$ prompt compression ratio, while retaining about 98% of the downstream performance of the original system prompts. This substantially reduces inference cost and frees nearly the entire context window for user inputs and model outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。