arXiv:2504.07952cs.LGcs.CL2025-04Conference of the …被引 111

让大模型在推理时自动积累经验,提升解题能力。

Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory

  • 引入动态记忆机制,在推理中持续存储和复用解题策略。
  • GPT-4o在24点游戏成功率从10%升至99%,数学题准确率翻倍。
  • 无需标注或人工反馈,适合需要长期积累的复杂任务

尽管当前语言模型在复杂任务上表现优异,但通常孤立处理每个输入,无法保留过往经验。本文提出动态笔记(Dynamic Cheatsheet, DC),一种轻量级框架,为黑盒语言模型赋予持续演化的记忆能力。模型可在推理时存储并复用积累的解题策略、代码片段与通用洞察,避免重复试错。该测试时学习显著提升性能,无需标注或人工反馈。使用DC后,Claude 3.5 Sonnet在AIME数学考试中的准确率超过翻倍;GPT-4o在24点游戏中成功率从10%提升至99%。在易出算术错误的任务如方程平衡中,通过回忆已验证代码,GPT-4o与Claude达到接近完美的准确率,而基线模型停滞在约50%。在知识密集型任务上也表现突出:Claude在GPQA-Diamond上提升9%,在MMLU-Pro上提升8%。关键在于,DC的记忆由模型自主筛选,聚焦简洁可迁移的代码片段,而非完整对话记录。不同于微调或静态检索,DC在不修改参数的前提下实时优化模型解题能力。结果表明,该方法为语言模型注入持久记忆,更贴近人类累积经验的学习模式。

原文摘要 · Abstract (English)

Despite their impressive performance on complex tasks, current language models (LMs) typically operate in a vacuum: Each input query is processed separately, without retaining insights from previous attempts. Here, we present Dynamic Cheatsheet (DC), a lightweight framework that endows a black-box LM with a persistent, evolving memory. Rather than repeatedly re-discovering or re-committing the same solutions and mistakes, DC enables models to store and reuse accumulated strategies, code snippets, and general problem-solving insights at inference time. This test-time learning enhances performance substantially across a range of tasks without needing explicit ground-truth labels or human feedback. Leveraging DC, Claude 3.5 Sonnet's accuracy more than doubled on AIME math exams once it began retaining algebraic insights across questions. Similarly, GPT-4o's success rate on Game of 24 increased from 10% to 99% after the model discovered and reused a Python-based solution. In tasks prone to arithmetic mistakes, such as balancing equations, DC enabled GPT-4o and Claude to reach near-perfect accuracy by recalling previously validated code, whereas their baselines stagnated around 50%. Beyond arithmetic challenges, DC yields notable accuracy gains on knowledge-demanding tasks. Claude achieved a 9% improvement in GPQA-Diamond and an 8% boost on MMLU-Pro problems. Crucially, DC's memory is self-curated, focusing on concise, transferable snippets rather than entire transcript. Unlike finetuning or static retrieval methods, DC adapts LMs' problem-solving skills on the fly, without modifying their underlying parameters. Overall, our findings present DC as a promising approach for augmenting LMs with persistent memory, bridging the divide between isolated inference events and the cumulative, experience-driven learning characteristic of human cognition.

推理增强动态记忆大模型测试时学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。