用元学习视角解析大模型推理轨迹，揭示其内在优化机制。
Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective
- 将推理过程类比为参数优化，构建元学习框架
- 在多样题目上训练后，模型可泛化到未见问题
- 为改进大模型提供可借鉴的元学习方法
我们提出一种新框架，从元学习视角理解大语言模型（LLM）的推理能力。通过将推理轨迹视为对模型参数的伪梯度下降更新，我们发现LLM推理与多种元学习范式存在相似性。将每个问题视为独立任务，推理轨迹作为内循环优化以适应模型参数，训练过程被形式化为元学习设置。在多样化问题上训练后，模型发展出可泛化至未见问题的基本推理能力。大量实证评估验证了LLM推理与元学习间的强关联，从元学习角度探讨了若干重要问题。本工作不仅深化了对LLM推理的理解，还为利用成熟元学习技术提升模型提供了实用洞见。
原文摘要 · Abstract (English)
We propose a novel framework for comprehending the reasoning capabilities of large language models (LLMs) through the perspective of meta-learning. By conceptualizing reasoning trajectories as pseudo-gradient descent updates to the LLM's parameters, we identify parallels between LLM reasoning and various meta-learning paradigms. We formalize the training process for reasoning tasks as a meta-learning setup, with each question treated as an individual task, and reasoning trajectories serving as the inner loop optimization for adapting model parameters. Once trained on a diverse set of questions, the LLM develops fundamental reasoning capabilities that can generalize to previously unseen questions. Extensive empirical evaluations substantiate the strong connection between LLM reasoning and meta-learning, exploring several issues of significant interest from a meta-learning standpoint. Our work not only enhances the understanding of LLM reasoning but also provides practical insights for improving these models through established meta-learning techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。