高效挑选推理题示范样本，减少模型调用并提升准确率
EXPLORA: Efficient Exemplar Subset Selection for Complex Reasoning
- 设计新方法估算评分函数，无需置信度信息筛选最优示例
- 仅需现有方法11%的LLM调用次数，性能提升12.24%
- 适合需要快速推理且对稳定性要求高的复杂问答场景
在文本与混合源（包括表格）上回答基于推理的复杂问题是一项挑战。近年来大语言模型（LLMs）的发展推动了上下文学习（ICL）的应用，使模型仅通过少量示范样本即可掌握特定任务。ICL中的关键难题在于如何选择最优示范样本，可分为任务特定（静态）或测试例特定（动态）两类。静态示例能实现更快的推理速度并在测试分布上更具鲁棒性。本文提出一种用于复杂推理任务的静态示例子集选择算法——EXPLORA。该方法引入一种新颖的探索机制，用于估计评分函数参数，以评估示例子集而无需依赖置信度信息。EXPLORA将所需LLM调用次数降低至现有最优方法的约11%，同时实现12.24%的显著性能提升。代码与数据已开源（https://github.com/kiranpurohit/EXPLORA）。
原文摘要 · Abstract (English)
Answering reasoning-based complex questions over text and hybrid sources, including tables, is a challenging task. Recent advances in large language models (LLMs) have enabled in-context learning (ICL), allowing LLMs to acquire proficiency in a specific task using only a few demonstration samples (exemplars). A critical challenge in ICL is the selection of optimal exemplars, which can be either task-specific (static) or test-example-specific (dynamic). Static exemplars provide faster inference times and increased robustness across a distribution of test examples. In this paper, we propose an algorithm for static exemplar subset selection for complex reasoning tasks. We introduce EXPLORA, a novel exploration method designed to estimate the parameters of the scoring function, which evaluates exemplar subsets without incorporating confidence information. EXPLORA significantly reduces the number of LLM calls to ~11% of those required by state-of-the-art methods and achieves a substantial performance improvement of 12.24%. We open-source our code and data (https://github.com/kiranpurohit/EXPLORA).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。