将提示词转化为可复用的权重更新，实现精准模型编辑
Transmuting prompts into weights
- 从提示词推导出与令牌无关的思想向量和矩阵
- 理论解释现有向量/矩阵编辑方法，支持新知识注入
- 适合研究模型可解释性与可控生成的学者
大量研究表明，通过直接修改大语言模型的内部状态，可在推理时有效控制其行为，例如通过激活向量加法或权重矩阵更新。这些技术虽强大，却常依赖经验启发式方法，如从对比提示的平均激活中提取“引导向量”。基于Dherin等（2025）的工作——发现提示的影响在数学上对应于依赖标记的隐式权重更新，并提出静态思维补丁概念用于提示压缩——本文将该框架提升为一种稳健的直接模型编辑算法。我们推导出一种原则性方法，将这种瞬态信息压缩为与令牌无关的思想向量和思想矩阵。这些构建不仅为现有基于向量和矩阵的模型编辑技术提供理论解释，还提供了直接、计算上可行的方法，将文本输入转化为复杂架构的可复用权重更新，适用于新知识注入。
原文摘要 · Abstract (English)
A growing body of research has demonstrated that the behavior of large language models can be effectively controlled at inference time by directly modifying their internal states, either through vector additions to their activations or through updates to their weight matrices. These techniques, while powerful, are often guided by empirical heuristics, such as deriving ``steering vectors'' from the average activations of contrastive prompts. Building on the foundational work of Dherin et al. (2025), who discovered that a prompt's influence mathematically maps to token-dependent implicit weight updates and introduced the initial concept of a static thought patch for prompt compression, we elevate this framework into a robust algorithm for direct model editing. We derive a principled method for condensing this transient information into token-independent thought vectors and thought matrices. These constructs provide a theoretical explanation for existing vector-and-matrix-based model editing techniques and offer a direct, computationally-grounded method for transmuting textual input into reusable weight updates for complex architectures and new knowledge injection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。