让大模型学会从上下文自发现任务规范,显著提升推理成功率。
Agentic Context Learning with Self-Discovered Specification

- 通过提取上下文中的隐含规则与格式,强制模型遵守本地规范。
- 在CL-Bench上实现28.14%任务成功率,较基线提升超5个百分点。
- 适合研究大模型推理能力、提示工程与自主学习的学者参考。
上下文学习是新兴的推理阶段任务,要求大模型从预训练未覆盖的复杂上下文中学习并应用新知识,当前顶尖模型成功率仍低于24%。本研究通过全面实证分析发现,失败主因并非内容获取困难,而是缺乏对局部规范的识别——这些规范如领域格式、局部规则和完整性条件常未在查询中明示,却分散于上下文之中。在31,592个评分项中,55.4%评估的是规范获取,仅22.6%关注内容获取;尽管76.7%的规范未在查询中提及,但95.5%可从上下文中追溯。为此,我们提出简单有效的干预PSCI(私有规范-契约归纳),通过对抗性检查与修复强制模型学习并遵守规范,在GPT-5.1上达成28.14%的最高准确率(+5.59绝对百分点,+24.8%相对提升),并在Qwen3.5-27B和Gemini 3 Pro上复现。17项消融实验进一步验证了任务规范的关键作用。结果表明,上下文学习的成功不仅依赖内容理解,更取决于规范识别与内化。
原文摘要 · Abstract (English)
Context learning is an emerging inference-time task where LLMs must learn and apply novel, task-specific knowledge from intricate contexts absent from pre-training; even frontier models score under 24% task success. In this work, we conduct a comprehensive empirical study to understand why this setting remains difficult. A natural hypothesis is that failures stem from content access; yet across twelve retrieval, reflection, and verification baselines on CL-Bench, an extensive context learning benchmark, we find limited gains over direct full-context prompting. Further failure analysis reveals a key finding: unlike typical long-context tasks such as long document understanding, context learning requires not only recovering local content but also acquiring local specifications that are often unspecified in the query but distributed across the context: domain-specific formats, local rules, and completeness conditions. Across all 31,592 rubric items, we find that 55.4% clearly evaluate specification acquisition, while only 22.6% evaluate content acquisition. Moreover, despite 76.7% of specifications being unspecified in the user query, 95.5% are traceable to the context, indicating these are learnable obligations rather than hidden requirements. To validate this diagnosis, we design a deliberately simple intervention PSCI (private specification-contract induction) which extracts local specifications and enforces them through adversarial checking and repair; PSCI achieves state-of-the-art 28.14% with GPT-5.1 (+5.59 pp absolute and +24.8% relative) on CL-Bench, replicated on Qwen3.5-27B (+5.28 pp) and Gemini 3 Pro (+6.17 pp). Seventeen ablations further isolate the role of task-specific specifications. Overall, our results suggest context learning hinges on not only content acquisition but also specification acquisition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。