将任务知识转化为可独立加载的外部模块，提升模型使用效率。
KV-Skill: Forging Expertise in the Model's Native Language
- 用轻量接口让冻结模型读取外部因子化操作符，实现知识分离。
- 文本转操作符后在10个基准上准确率最高达77.2%，显著优于原始文本和现有方法。
- 一个共享接口可同时管理多个技能，无遗忘且可独立部署。
任务知识通常以提示文本或权重更新形式存储。文本灵活但需每次解析，权重修改则难以独立加载、移除或共享。我们提出KV-Skill，一种外部因子化算子的设计空间，让冻结的语言模型通过轻量接口读取。该方法支持两种路径：注册将文本技能转为文本衍生算子，并训练共享的每骨干接口；奖励学习则直接从任务结果生成紧凑潜在算子，无需人工编写技能。两种路径均不增加提示位置。在三个模型家族的四个主干模型上，十项基准测试中，文本转KV-Skill始终提升效果。Qwen3.5-4B LiveMath上，注册法达77.2%准确率，远超源文本技能（23.4%）、SkillOpt（52.0%）和SoftSkill（64.5%）。在相同奖励训练与参数预算下，8组对比中7组优于软前缀、前缀调优和LoRA。事后秩分析显示，每注入层仅需一个任务对齐方向即可保留近全部收益，随机方向则失效。单一共享接口可无损容纳三个独立可加载的KV-Skills。结果表明，任务知识可由文本或经验获取，压缩为外部算子，独立于主干模型部署。代码已开源。
原文摘要 · Abstract (English)
Task knowledge is commonly stored either as text in the prompt or as an update to model weights. Text is modular but must be interpreted on every use, while weight adaptation makes the resulting capability difficult to load, remove, or share independently. We introduce KV-Skill, a design space of external factorized operators that a frozen language model reads through a lightweight interface. KV-Skill supports two complementary paths. Registration converts an authored text skill into a text-derived operator and trains a shared per-backbone interface. Reward learning develops a compact latent operator directly from task outcomes, with or without an authored skill. Neither path adds positions to the prompt. Across ten benchmarks and four backbones from three model families, converting text to a KV-Skill consistently makes the same procedural knowledge more effective. On Qwen3.5-4B LiveMath, registration reaches 77.2 accuracy, compared with 23.4 for the source text skill, 52.0 for SkillOpt, and 64.5 for SoftSkill. Under matched reward training and parameter budgets, KV-Skill gives the best result in seven of eight matched settings against soft prefixes, prefix tuning, and LoRA. A post-hoc rank analysis further shows that text-derived operators retain nearly all of their benefit with one task-aligned direction per injection layer, while matched random directions fail. Finally, one shared interface retains three independently loadable KV-Skills without measurable forgetting. These results show that task knowledge can be acquired from text or experience, compressed into an external operator, and deployed separately from the backbone. Code is available at: https://github.com/shawnzhg/KV-Skill
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。