arXiv:2604.27660cs.AI2026-04被引 20

让大模型自动学会从上下文中提炼技能,无需人工标注。

From Context to Skills: Can Language Models Learn from Context Skillfully?

论文配图:From Context to Skills: Can Language Models Learn from Context Skillfully?
图 1 · 摘自论文原文
  • 用多智能体自博弈机制自动发现和优化上下文技能。
  • 在四个任务上提升大模型的解题率,最高达37.6%。
  • 适合想提升模型上下文学习能力的研究者与工程师。

现实任务常要求语言模型处理超出其参数知识的复杂上下文,这需要上下文学习能力。传统方法依赖人工构建自然语言技能,但成本高且缺乏反馈机制。本文提出Ctx2Skill,一种无需人工监督或外部反馈的自进化框架。其核心为多智能体自博弈循环:挑战者生成测试任务与标准,推理者基于不断演化的技能集尝试解答,裁判提供二元反馈。双方通过专门的提议与生成代理分析失败案例,自动更新技能。为防止任务极端化和技能过拟合,引入跨时间回放机制,筛选出在代表性任务上表现均衡的技能组合,确保泛化性。最终生成的技能可无缝接入任意语言模型,显著提升其上下文学习能力。在CL-bench的四个任务上评估,所有基线模型的解题率均持续提升,最高提升37.6%。

原文摘要 · Abstract (English)

Many real-world tasks require language models (LMs) to reason over complex contexts that exceed their parametric knowledge. This calls for context learning, where LMs directly learn relevant knowledge from the given context. An intuitive solution is inference-time skill augmentation: extracting the rules and procedures from context into natural-language skills. However, constructing such skills for context learning scenarios faces two challenges: the prohibitive cost of manual skill annotation for long, technically dense contexts, and the lack of external feedback for automated skill construction. In this paper, we propose Ctx2Skill, a self-evolving framework that autonomously discovers, refines, and selects context-specific skills without human supervision or external feedback. At its core, a multi-agent self-play loop has a Challenger that generates probing tasks and rubrics, a Reasoner that attempts to solve them guided by an evolving skill set, and a neutral Judge that provides binary feedback. Crucially, both the Challenger and the Reasoner evolve through accumulated skills: dedicated Proposer and Generator agents analyze failure cases and synthesize them into targeted skill updates for both sides, enabling automated skill discovery and refinement. To prevent adversarial collapse caused by increasingly extreme task generation and over-specialized skill accumulation, we further introduce a Cross-time Replay mechanism that identifies the skill set achieving the best balance across representative cases for the Reasoner side, ensuring robust and generalizable skill evolution. The resulting skills can be plugged into any language model to obtain better context learning capability. Evaluated on four context learning tasks from CL-bench, Ctx2Skill consistently improves solving rates across backbone models.

上下文学习多智能体自进化技能提取

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。