让大模型学会高效提问，用贝叶斯方法提升多轮对话中的信息获取能力。
Amortising Bayesian Experimental Design for Sequential Information Gathering in LLMs
- 将贝叶斯实验设计转化为可复用的策略，通过奖励机制训练大模型
- 在20个问题任务中成功率超基线模型一倍以上，推理成本降低25倍
- 策略可迁移至医疗诊断等未见场景，适合需要精准提问的应用
大型语言模型虽具备强大推理与世界知识能力，但在多轮交互中获取信息效率较低。本文提出一种名为ASIG的微调方法，将贝叶斯实验设计（BED）通过多轮分组相对策略优化，转化为可复用的模型策略，并引入期望信息增益作为奖励信号。在20个问题任务中，7B规模基线模型的成功率被提升超过一倍，且推理成本较竞争性推理时基线BED-LLM降低25倍以上。该方法应用于训练阶段未见过的医疗诊断基准MediQ，仍显著提升信息获取性能，表明所学策略具备良好的跨域泛化能力。结果表明，将贝叶斯实验设计融入大模型策略是实现高效序列信息收集的有效且计算高效的方案。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit strong reasoning and world-knowledge capabilities, yet often struggle to gather information effectively across the multi-turn interactions required in sequential decision-making settings. We introduce Amortised Sequential Information Gathering (ASIG), a fine-tuning approach that amortises Bayesian Experimental Design (BED) into LLM policies via a multi-turn extension of Group Relative Policy Optimisation with an Expected Information Gain reward. Evaluated on the 20 Questions task, ASIG more than doubles the success rate of the 7B base model and reduces inference cost by over $25\times$ relative to BED-LLM, a competitive inference-time baseline. Applied to MediQ, a medical diagnosis benchmark unseen during training, ASIG improves information-seeking performance at the 7B scale, suggesting that the learned strategies can transfer out of distribution. Our findings show that amortising BED into LLM policies provides an effective and computationally efficient approach to sequential information gathering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。