arXiv:2607.23304stat.MLcs.LG2026-07

统一三类自适应推理方法,揭示其数学本质并提出实用设计准则。

Context-Adaptive Inference: A Unified Statistical and Foundation-Model View

  • 将统计、元学习与大模型隐式适应统一为上下文映射参数的框架
  • 证明显式参数调整与隐式路由在特定条件下等价于核岭回归
  • 提出效率、稳定性与鲁棒性评估指标,指导实际部署中的自适应决策

现代预测系统需根据具体情境调整行为:临床模型不应对所有患者一视同仁;检索增强模型在不同证据下应生成不同答案;混合专家模型应将不同输入路由至不同专家。这种能力称为上下文自适应推理——在预测前利用当前上下文信息,针对该实例特化参数或计算。本文从三个传统出发提供统一视角:(i) 统计中的显式适应(如变系数模型、局部回归、层次共享),(ii) 元学习与迁移中的快速任务适应,(iii) 大型基础模型通过提示、检索和专家路由实现的隐式适应。我们形式化这些方法为同一目标:将上下文 $c$ 映射到适配参数 $θ(c)$,再通过 $f(x; θ(c))$ 预测。在平方损失、线性预测头和固定特征下,证明显式参数适应与隐式路由数学上等价于输入与上下文联合特征上的核岭回归。基于此桥梁,提出实用设计原则与评估指标,包括适应效率、路由稳定性与上下文特定鲁棒性,指导何时特化、如何约束特化以及如何审计部署中的上下文自适应模型。最后,指出可识别性、分布偏移下的鲁棒性与大规模高效适应等开放问题,提出可扩展、可靠、透明的方法设计原则。

原文摘要 · Abstract (English)

Modern predictive systems are expected to adapt their behavior to the specific situation they are facing. A clinical model should not treat every patient the same; a retrieval-augmented model should change its answer when given different evidence; a mixture-of-experts model should route different inputs to different experts. We call this capability context-adaptive inference: before predicting, the system uses information about the current context to specialize its parameters or computation for that instance. This article provides a unified view of context-adaptive inference across three traditions that are usually treated separately: (i) explicit adaptation in statistics (e.g. varying-coefficient models, local regression, hierarchical sharing), (ii) rapid task-specific adaptation in meta-learning and transfer, and (iii) implicit adaptation in large foundation models via prompting, retrieval, and expert routing. We formalize these approaches under a common objective: to map context $c$ to adapted parameters $θ(c)$, then to predict via $f(x; θ(c))$. Under squared loss, linear prediction heads, and fixed features, we prove that explicit parameter adaptation and implicit routing are mathematically equivalent to kernel ridge regression on joint features of inputs and context. Building on this bridge, we propose practical design principles and evaluation metrics including adaptation-efficiency, routing stability, and context-specific robustness to guide when to specialize, how to constrain that specialization, and how to audit context-adaptive models in deployment. Finally, we identify open problems in identifiability, robustness under distribution shift, and efficient large-scale adaptation, outlining design principles for methods that are scalable, reliable, and transparent in real-world settings.

自适应推理基础模型元学习统计建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。