让大模型根据问题动态选推理策略，提升问答效果并减少资源消耗
Dynamic Strategy Planning for Efficient Question Answering with Large Language Models
- 根据问题类型自动选择推理、规划或检索策略
- 在三个多跳问答数据集上性能提升7%-13%，成本降低11%-32%
- 适合追求高效高精度问答系统的开发者与研究者
研究表明，推理（如思维链）、规划（如SelfAsk）和检索增强生成等策略能有效提升大语言模型在问答等任务上的表现。然而，对所有问题使用单一固定策略会降低性能，且在生成文本长度和检索次数上效率低下。本文提出DyPlan新方法，使大模型具备动态策略选择能力，根据输入问题决定最优策略，并据此引导生成过程。进一步提出DyPlan-verify，在生成中加入内部验证与修正机制以增强答案质量。在三个主流多跳问答数据集上的实验表明，DyPlan相比最佳基线模型，性能提升7%-13%，成本降低11%-32%。
原文摘要 · Abstract (English)
Research has shown the effectiveness of reasoning (e.g., Chain-of-Thought), planning (e.g., SelfAsk), and retrieval augmented generation strategies to improve the performance of Large Language Models (LLMs) on various tasks, such as question answering. However, using a single fixed strategy to answer different kinds of questions is suboptimal in performance and inefficient in terms of generated output tokens and performed retrievals. In our work, we propose a novel technique DyPlan, to induce a dynamic strategy selection process in LLMs, to improve performance and reduce costs in question-answering. DyPlan incorporates an initial decision step to select the most suitable strategy conditioned on the input question and guides the LLM's response generation accordingly. We extend DyPlan to DyPlan-verify, adding an internal verification and correction process to further enrich the generated answer. Experiments on three prominent multi-hop question answering (MHQA) datasets reveal how DyPlan can improve model performance by 7-13% while reducing the cost by 11-32% relative to the best baseline model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。