用不同提问方式比换模型更能提升大模型问答准确率
Diverse LLMs or Diverse Question Interpretations? That is the Ensembling Question
- 同一模型用多种问法提问，再投票决定答案
- 在三个数据集上，新方法准确率均高于换模型方案
- 适合想提升问答效果但不想多用模型的开发者
有效利用多样性已被证明可提升各类机器学习模型的性能，包括大语言模型（LLMs）。然而，如何最有效地运用多样性仍是一个挑战。本文对比了两种用于回答二元问题的多样性策略：模型多样性（多个模型回答同一问题）和问题解释多样性（同一模型回答不同表述的问题）。两种情况下均采用多数投票作为集成共识机制。在BoolQ、StrategyQA和PubMedQA上的实验表明，问题解释多样性在集成准确率上始终优于模型多样性。此外，对GPT和LLaMa的分析显示，模型多样性通常仅产生介于最佳与最差成员之间的结果，且无明显提升。
原文摘要 · Abstract (English)
Effectively leveraging diversity has been shown to improve performance for various machine learning models, including large language models (LLMs). However, determining the most effective way of using diversity remains a challenge. In this work, we compare two diversity approaches for answering binary questions using LLMs: model diversity, which relies on multiple models answering the same question, and question interpretation diversity, which relies on using the same model to answer the same question framed in different ways. For both cases, we apply majority voting as the ensemble consensus heuristic to determine the final answer. Our experiments on boolq, strategyqa, and pubmedqa show that question interpretation diversity consistently leads to better ensemble accuracy compared to model diversity. Furthermore, our analysis of GPT and LLaMa shows that model diversity typically produces results between the best and the worst ensemble members without clear improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。