对比了固定与自适应查询在任务近似中的表现，发现表示约束会改变自适应的优势。
Adaptivity Under Realizability Constraints: Comparing In-Context and Agentic Learning
- 用固定和自适应查询比较任务族的统一近似能力
- 在可实现条件下，自适应优势可能消失或仅在特定场景出现
- 揭示了神经网络表示限制对自适应学习的关键影响
我们比较了使用固定查询的上下文学习与使用自适应查询的代理学习，在任务族的统一近似能力方面。考虑两种设置：无限制情形（查询与近似为任意函数）和可实现情形（要求由ReLU神经网络实现）。在两种设置中，自适应性均不会损害近似性能。然而，从无限制到可实现设置的转变会使这种优势发生变化。我们识别出四种不同的近似情景，每种均由一个明确的任务族所体现：(a) 自适应无优势；(b) 在无限制下有优势，且在ReLU可实现下仍保持；(c) 仅在可实现下产生优势；(d) 在可实现下优势消失。这表明表示约束与自适应效应之间存在深刻互动。
原文摘要 · Abstract (English)
We compare in-context learning with fixed queries and agentic learning with adaptive queries for uniform approximation of task families. We consider two settings: an unrestricted regime, where querying and approximation are arbitrary functions, and a realizable regime, where we require these operations to be implemented by ReLU neural networks. In both settings, adaptivity never hinders approximation performance. However, this advantage can change when one passes from the unrestricted regime to the realizable regime. We identify four distinct approximation scenarios, each witnessed by an explicit task family: (a) no advantage of adaptivity; (b) an advantage in the unrestricted regime that persists under ReLU realizability; (c) an advantage that arises only under realizability; and (d) an advantage that disappears under realizability. This demonstrates that representational constraints interact profoundly with the effect of adaptivity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。