arXiv:2510.13905cs.CLcs.AI2025-10被引 4

用认知框架提升大模型推理,让少样本学习更准更可解释

Schema for In-Context Learning

论文配图:Schema for In-Context Learning
图 1 · 摘自论文原文
  • 提取先验示例中的推理结构,构建轻量抽象框架
  • 在高质量单例下性能最高提升36.19%,减少对示例数量依赖
  • 适合需要类人逻辑推理与可解释性的任务场景

上下文学习(ICL)使基于Transformer的语言模型通过示范样例适应新任务。然而,传统方法缺乏在抽象层面显式进行知识检索与迁移的模块。受认知科学中图式理论启发,人类通过激活已有心智框架来组织理解新信息,我们提出图式激活式上下文学习(SA-ICL)。该框架从先前示例中提取认知构建块的表示,生成一个轻量级、结构化的关键推理步骤及其关系模板——即抽象图式,并用于增强模型面对新问题时的推理过程。实验表明,多数大语言模型无法隐式形成和使用图式化学习表征,但通过显式图式支架显著获益。在GPQA数据集的化学与物理问题上,当仅有一个高质量示范例时,SA-ICL性能最高提升36.19%,同时降低对示例数量的依赖并提升可解释性。该方法不仅融合了从模式提示到思维链提示的多种ICL策略,也为提升大模型类人推理能力开辟新路径。

原文摘要 · Abstract (English)

In-Context Learning (ICL) enables transformer-based language models to adapt to new tasks by conditioning on demonstration examples. However, traditional example-driven in-context learning lacks explicit modules for knowledge retrieval and transfer at the abstraction level. Inspired by cognitive science, specifically schema theory, which holds that humans interpret new information by activating pre-existing mental frameworks (schemas) to structure understanding, we introduce Schema-Activated In-Context Learning (SA-ICL). This framework extracts the representation of the building blocks of cognition for the reasoning process instilled from prior examples, creating an abstracted schema, a lightweight, structured template of key inferential steps and their relationships, which is then used to augment a model's reasoning process when presented with a novel question. We demonstrate that a broad range of large language models (LLMs) lack the capacity to form and utilize internal schema-based learning representations implicitly, but instead benefit significantly from explicit schema-based scaffolding. Across chemistry and physics questions from the GPQA dataset, our experiments show that SA-ICL consistently boosts performance, up to 36.19 percent, when the single demonstration example is of high quality, which simultaneously reduces reliance on the number of demonstrations and enhances interpretability. Schema-Activated In-Context Learning not only bridges disparate ICL strategies ranging from pattern priming to Chain-of-Thought prompting, but also paves a new path for enhancing human-like reasoning in LLMs.

上下文学习推理增强可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。