将文本技能转为可参数化的模型权重,让大模型更稳定地使用复杂技能。
Parametric Skills

- 用超网络把文本技能转化为LoRA参数,在推理时直接调用。
- 在6个软件工程任务上平均比传统提示学习高6.44分。
- 支持技能累积,有望实现测试时持续学习,适合长期智能体应用。
由于智能本质上依赖于高效的能力获取(Chollet, 2019),利用能力的能力至关重要。对于大语言模型而言,能力是通过人工编写或从任务轨迹中提取的文本化方案,编码了成熟的问题解决经验,对智能体能力至关重要。尽管广泛应用,其效用受限于模型理解并遵循技能指令的能力,尤其在复杂、长上下文场景下,关键指令难以定位与执行。为此,我们提出ParametricSkills框架,可在推理时将自由形式的文本技能转换为参数,实现无上下文依赖的能力利用。具体而言,我们首先构建大规模高质量技能库,并使用OpenCode合成围绕这些技能的单轮与多轮技能执行轨迹。基于这些数据,训练一个超网络,接收文本技能并将其转化为LoRA适配器,从而参数化技能内容与推理方法。在六个复杂软件工程子任务上的实验表明,所提方法在DeepSeek-V4-Flash评估下,平均性能比上下文学习高出6.44分,同时显著提升BERT Score与F1分数,验证了其有效性。此外,我们发现参数化技能具有天然可累积性,为测试时持续学习提供了初步但有前景的方向。
原文摘要 · Abstract (English)
Since intelligence fundamentally relies on efficient skill acquisition (Chollet, 2019), the ability to leverage skills is critical. For LLMs, skills, manually authored or extracted from task trajectories, are textual recipes encoding mature problem-solving experience and are critical to agentic capabilities. Despite widespread deployment, their utility is limited by the model's ability to comprehend and follow skill instructions, especially under complex and long-context scenarios, where key instructions are difficult to locate and adhere to. To address this limitation, we propose ParametricSkills, a framework that can convert free-form textual skills into parameters at test time, enabling context-free skill exploitation. Specifically, we first construct a large-scale, high-quality skill library, and synthesize single-turn and multi-turn skill exploitation trajectories built around these skills with OpenCode. Using these data, we then train a hypernetwork that parameterizes both the skill content and the test-time exploitation methodology by receiving textual skills and converting them into LoRA adapters. Experimental results on six complex software engineering (SWE) subtasks demonstrate that, the proposed ParametricSkills averagely outperforms in-context learning by 6.44 points as judged by DeepSeek-V4-Flash, while also achieving significantly higher BERT Score and F1 score, confirming its effectiveness. Beyond performance, we further find that parametric skills, being inherently accumulative, offer a preliminary yet promising avenue toward test-time continual learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。