用单个可训练标记实现任务记忆，让大模型高效复用指令。
TokMem: One-Token Procedural Memory for Large Language Models
- 将可复用的任务编译为单个可训练记忆标记，作为生成控制信号。
- 在1000个任务上表现优于检索增强提示，且无重复上下文开销。
- 参数量极少，适合快速添加新任务，不干扰已有知识。
大型语言模型通常依赖提示进行控制，每次新查询都需重新处理提示，难以模块化复用。我们提出TokMem，一种过程记忆框架，将每个可复用的任务流程编译为一个可训练的记忆标记。每个标记既是流程索引，也是生成控制信号，能以固定开销引导模型产生目标行为。TokMem保持基础LLM冻结，将过程知识完全存储于专用标记中，支持持续添加新流程而不影响已有内容。我们在两个场景下评估：在1,000个Super-Natural Instructions任务上进行原子召回，以及多步函数调用的组合召回。结果表明，TokMem在性能上持续优于检索增强提示，且避免重复上下文开销；同时在效果上达到或超越参数高效微调，但所用可训练参数显著更少。
原文摘要 · Abstract (English)
Large language models are typically controlled via prompts, which must be repeatedly re-processed for every new query and are difficult to reuse modularly. We introduce TokMem, a procedural memory framework that compiles each reusable task procedure into a single trainable memory token. Each token serves as both a procedure index and a generation control signal that steers generation, enabling targeted behaviors with constant-size overhead. TokMem keeps the backbone LLM frozen and stores procedural knowledge entirely in these dedicated units, so new procedures can be added continually without interfering with existing ones. We evaluate TokMem on two settings: atomic recall over 1,000 Super-Natural Instructions tasks and compositional recall on multi-step function-calling. Our results show that TokMem consistently outperforms retrieval-augmented prompting while avoiding repeated context overhead. Moreover, it matches or exceeds parameter-efficient fine-tuning with substantially fewer trainable parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。