让大模型主动查资料,提升任务表现且无需重训练。
Context Training with Active Information Seeking

- 用搜索工具动态优化上下文,实现无权重更新的高效适配。
- 在翻译、医疗和推理任务上显著优于传统方法,最高提升17%。
- 适合资源有限但需快速响应新知识的场景,如医疗问答或小众领域。
现有大语言模型(LLMs)在部署后难以低成本适配新任务,尤其涉及新信息或冷门领域时。近期研究发现,仅通过优化上下文即可实现任务定制而无需更新模型权重。然而,多数方法仍依赖模型自身知识,缺乏外部信息获取能力。本文为上下文优化器引入维基百科搜索与浏览器工具,实现主动信息检索。我们发现,简单地将工具加入序列化优化流程反而会降低性能。但若采用基于搜索的训练机制,持续维护并修剪多个候选上下文,则能带来稳定且显著的提升。实验覆盖低资源翻译(Flores+)、医疗场景(HealthBench)及强推理任务(LiveCodeBench 和 Humanity's Last Exam),均取得明显改进。此外,该方法数据效率高,对超参数鲁棒,生成的上下文可跨模型良好泛化。
原文摘要 · Abstract (English)
Most existing large language models (LLMs) are expensive to adapt after deployment, especially when a task requires newly produced information or niche domain knowledge. Recent work has shown that, by manipulating and optimizing their context, LLMs can be tailored to downstream tasks without updating their weights. However, most existing methods remain closed-loop, relying solely on the model's intrinsic knowledge. In this paper, we equip these context optimizers with Wikipedia search and browser tools for active information seeking. We show that naively adding these tools to a standard sequential context optimization pipeline can actually degrade performance compared to baselines. However, when paired with a search-based training procedure that maintains and prunes multiple candidate contexts, active information seeking delivers consistent and substantial gains. We demonstrate these improvements across diverse domains, including low-resource translation (Flores+), health scenarios (HealthBench), and reasoning-heavy tasks (LiveCodeBench and Humanity's Last Exam). Furthermore, our method proves to be data-efficient, robust across different hyperparameters, and capable of generating effective textual contexts that generalize well across different models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。