arXiv:2412.13422cs.AIcs.SE2024-12NAACL被引 12

提升大模型归纳推理的假设多样性,避免重复浪费算力。

Generating Diverse Hypotheses for Inductive Reasoning

  • 用概念混合机制模拟人类推理,生成更丰富的假设
  • 在多个基准上显著优于标准采样方法,准确率更高
  • 适合需要多角度推理的AI研究与应用开发者

归纳推理——从少量观察中推断普遍规律——是人类智能的核心。近期研究表明,大语言模型(LLMs)可通过采样多个规则假设并选择最能解释观测结果的来实现归纳推理。然而,由于独立同分布(IID)采样,常生成语义重复的假设,造成大量计算资源浪费。本文首先表明,提高温度以增加多样性受限于文本退化问题;其次提出一种新方法,受人类归纳推理启发,称为概念混合(Mixture of Concepts, MoC)。分析显示,温度升高虽短期内提升假设多样性和准确性,但因文本退化而趋于饱和。在多个归纳推理基准测试中,MoC相比标准IID采样及其他方法均表现出显著性能提升。

原文摘要 · Abstract (English)

Inductive reasoning - the process of inferring general rules from a small number of observations - is a fundamental aspect of human intelligence. Recent works suggest that large language models (LLMs) can engage in inductive reasoning by sampling multiple hypotheses about the rules and selecting the one that best explains the observations. However, due to the IID sampling, semantically redundant hypotheses are frequently generated, leading to significant wastage of compute. In this paper, we 1) demonstrate that increasing the temperature to enhance the diversity is limited due to text degeneration issue, and 2) propose a novel method to improve the diversity while maintaining text quality. We first analyze the effect of increasing the temperature parameter, which is regarded as the LLM's diversity control, on IID hypotheses. Our analysis shows that as temperature rises, diversity and accuracy of hypotheses increase up to a certain point, but this trend saturates due to text degeneration. To generate hypotheses that are more semantically diverse and of higher quality, we propose a novel approach inspired by human inductive reasoning, which we call Mixture of Concepts (MoC). When applied to several inductive reasoning benchmarks, MoC demonstrated significant performance improvements compared to standard IID sampling and other approaches.

归纳推理大模型假设生成多样性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。