让大模型在推理时自动学会更高效地扩展计算,像人脑一样分时段学习和巩固。
LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space
- 模仿人脑双系统,白天快速调用历史隐状态,夜晚缓慢整合优化经验。
- 在8个基准上比现有方法最高提升13.33%,跨模型和跨领域表现优异。
- 无需标注数据,完全自监督,适合追求推理性能的AI研究者。
测试时扩展(TTS)已被证明能在不修改模型参数的情况下显著提升大语言模型(LLM)的推理能力。然而,现有TTS方法大多独立运行,导致LLM尚未能逐步学会如何更有效地进行扩展。为使LLM学会‘如何在测试时扩展计算’,我们提出LatentEvolve,一种受互补学习系统(CLS)理论启发的自演化隐空间TTS框架。该框架包含两个演化组件:白天扩展(daytime scaling),快速检索历史隐状态以更好引导当前推理;夜间扩展(nighttime scaling),类似人脑睡眠时的经验巩固,整合过往隐空间优化。昼夜交替机制实现了对LLM TTS的快速与缓慢演化,完全无监督地模拟人类认知动态。在八个基准和五个模型骨干上的实验表明,LatentEvolve相比最先进的方法(如LatentSeek和TTRL)最高提升达13.33%,展现出出色的跨领域与跨模型泛化能力。
原文摘要 · Abstract (English)
Test-time Scaling (TTS) has been demonstrated to significantly enhance the reasoning capabilities of Large Language Models (LLMs) during the inference phase without altering model parameters. However, existing TTS methods are largely independent, implying that LLMs have not yet evolved to progressively learn how to scale more effectively. With the objective of evolving LLMs to learn ``how to scale test-time computation,'' we propose LatentEvolve, a self-evolving latent TTS framework inspired by the complementary learning system (CLS) theory. Analogous to the human brain's dual system of a fast-recall hippocampus and a slow-consolidating neocortex, LatentEvolve comprises two evolutionary components: \textit{daytime scaling}, which rapidly retrieves historical latent representations to better guide current LLM reasoning; and \textit{nighttime scaling}, which integrates past latent optimizations in a manner akin to the human brain's consolidation of experiences during sleep. The alternation of daytime and nighttime processes facilitates a fast and slow evolution of LLM TTS, mirroring human cognitive dynamics in a fully unsupervised manner. Extensive experiments across eight benchmarks and five model backbones demonstrate that our LatentEvolve surpasses state-of-the-art TTS methods such as LatentSeek and TTRL by up to $13.33\%$ and exhibits exceptional cross-domain and cross-backbone generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。