arXiv:2410.02892cs.AIcs.CL2024-10ACL被引 27

通过动态融合演绎与归纳推理,提升大模型在复杂任务中的表现。

The Role of Deductive and Inductive Reasoning in Large Language Models

  • 引入双指标评估体系,根据任务难度动态调整推理路径。
  • 在AIW数据集上达70.3%准确率,优于Tree of Thought的62.2%。
  • 适合研究认知对齐、高效推理的大模型开发者使用。

大型语言模型(LLMs)在推理任务中展现出强大能力,但其对固定提示结构的依赖及在复杂场景下的适应性不足仍是主要挑战。本文提出一种名为演绎与归纳(DID)的新框架,通过动态融合演绎与归纳推理,增强模型的推理能力。基于认知科学原理,DID采用结合Littlestone维数与信息熵的双指标复杂度评估系统,精准判断任务难度并引导分解策略。该方法使模型能根据问题复杂度逐步调整推理路径,模拟人类认知过程。我们在多个基准测试中评估了DID的有效性,包括AIW和MR-GSM8K,以及我们自建的用于时序推理的Holiday Puzzle数据集。结果表明,DID在推理质量与解题准确率上均有显著提升,在AIW上达到70.3%的准确率(相较Tree of Thought的62.2%),同时保持较低计算开销。DID的成功验证了其在提升模型性能的同时维持计算效率的潜力,为构建更贴近人类认知、具备更强能力的模型提供了理论支持与高效替代方案。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning tasks, yet their reliance on static prompt structures and limited adaptability to complex scenarios remains a significant challenge. In this paper, we propose the Deductive and InDuctive(DID) method, a novel framework that enhances LLM reasoning by dynamically integrating both deductive and inductive reasoning approaches. Drawing from cognitive science principles, DID implements a dual-metric complexity evaluation system that combines Littlestone dimension and information entropy to precisely assess task difficulty and guide decomposition strategies. DID enables the model to progressively adapt its reasoning pathways based on problem complexity, mirroring human cognitive processes. We evaluate DID's effectiveness across multiple benchmarks, including the AIW and MR-GSM8K, as well as our custom Holiday Puzzle dataset for temporal reasoning. Our results demonstrate significant improvements in reasoning quality and solution accuracy - achieving 70.3% accuracy on AIW (compared to 62.2% for Tree of Thought) while maintaining lower computational costs. The success of DID in improving LLM performance while preserving computational efficiency suggests promising directions for developing more cognitively aligned and capable language models. Our work contributes a theoretically grounded, input-centric approach to enhancing LLM reasoning capabilities, offering an efficient alternative to traditional output-exploration methods.

大模型推理认知对齐动态路径

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。