arXiv:2607.03441cs.LGcs.AI2026-07

让大模型在运行时持续自我优化,防止长期任务中能力退化。

No Time Like the Present: Agentic Test-Time Training for LLM Agents

  • 基于令牌重加权实现连续测试时训练,抑制重复文本带来的错误迭代。
  • 在ALFWorld和SWE-bench Lite上分别提升5.0和4.9点成功率。
  • 适合需要长时间稳定表现的智能体任务,尤其保护已有能力不退化。

大模型智能体在长周期任务中常出现性能下降:轨迹增长导致状态重复、动作无效重试、策略失效。测试时训练(TTT)可动态调整模型权重以适应任务变化,但现有方法多仅单次适配固定输入。本文研究多轮对话中连续TTT机制,其中每次更新改变后续生成文本的策略,形成自训练循环。当新信息出现时有益,但若智能体陷入死循环则会放大偏差。我们发现更新文本的重复性可区分两种状态,提出代理测试时训练(aTTT),一种基于词元级别的重加权方法:对先前更新中重复出现的n-gram令牌降低损失权重,保留新出现令牌的完整权重。为支持实时运行,构建基于vLLM运行时LoRA API的并发服务系统,开销仅增加1.9倍。aTTT在ALFWorld上提升最高5.0分,在SWE-bench Lite上提升4.9分。增益集中于已有任务能力但易随轨迹退化的场景,表明aTTT主要作用是维持现有能力而非学习新技能。

原文摘要 · Abstract (English)

LLM agents often degrade over long episodes: as trajectories grow, they revisit explored states, repeat failed actions, and lose strategies that previously worked. Test-time training (TTT) offers a way to adapt model weights to the evolving task state, but existing LLM TTT methods largely adapt once to a fixed input. We study continuous TTT in multi-turn agent episodes, where each update changes the policy that generates later training text. This creates a self-training loop that helps when new trajectory information appears, but can amplify drift when the agent gets stuck and repeatedly trains on similar text. We find that update-text repetition distinguishes these regimes and introduce Agentic Test-Time Training (aTTT), a token-level reweighting method that downweights the loss on tokens appearing in repeated $n$-grams from prior updates while leaving novel tokens fully weighted. To run such updates inside live episodes, we build a concurrent serving system using vLLM's runtime LoRA API, limiting overhead to 1.9$\times$ the no-TTT cost. aTTT improves success by up to 5.0 points on ALFWorld and 4.9 points on SWE-bench Lite. The gains concentrate where models already have task competence but drift over long trajectories, suggesting that aTTT mainly preserves existing competence rather than teaching new abilities.

大模型智能体测试时训练持续学习自训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。