提出有限记忆测试框架,隔离分析长时决策中记忆层的作用。
AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents

- 用类型化检索构建新提示,每步决策从干净消息开始,避免上下文混杂。
- 在《弑神者2》中,启用策略技能层后胜率从30%提升至60%。
- 适合研究长周期智能体记忆机制的开发者和研究人员使用。
长周期大模型智能体的记忆管理是一份关于未来决策可见内容的契约。传统方法将所有历史观测、工具调用和反思直接拼接至提示,导致上下文混乱且难以分离单个记忆成分的影响。本文提出一种受限契约:每步决策基于类型化检索组装的新用户消息,不附加原始跨决策记录,确保提示长度始终受限,可独立消融任一层次。我们在《弑神者2》(一个需数百次战术与战略决策的封闭规则卡牌构建游戏)中实现该框架。公开基准显示,前沿大模型在最低难度下五种配置均未获胜,人类在相同难度下胜率为16%,任务具挑战性但尚未饱和。固定A0消融实验表明,启用策略技能层时表现提升最大:无存储基线胜3/10场,加入技能层胜6/10场(样本量下方向性显著,Fisher精确检验p≈0.37)。跨模型探测及公开累积上下文基线作为操作对比,非对契约变量的受控测试。我们发布可复现的测试平台:298条完整轨迹带条件标签、冻结的记忆/技能快照、提示记录与分析脚本,提供智能体设计与验证方法论,支持研究显式记忆层如何影响长周期决策。
原文摘要 · Abstract (English)
Memory for a long-horizon LLM agent is a contract about what each future decision is allowed to see. The simplest contract appends past observations, tool calls, and reflections to every prompt, which makes prior context easy to access but also turns it into a jumbled mixture in which the effect of any single memory component is hard to isolate. We introduce and instrument an alternative bounded contract: every decision is made from a fresh user message assembled by typed retrieval, with no raw cross-decision transcript appended. The prompt thus stays bounded across runs of any length, and any single layer can be ablated in isolation. We instantiate the contract in Slay the Spire 2, a closed-rule stochastic deck-building game whose runs require hundreds of tactical and strategic decisions. A public online benchmark of frontier LLMs on the same game reports zero wins at the lowest difficulty across five configurations, and the developer-reported human win rate at the same difficulty is 16%; the task is hard but not saturated. Within our harness, a fixed-A0 ablation shows the largest observed difference when triggered strategic skills are enabled: the no-store baseline wins 3/10 games and adding the skill layer 6/10. At this sample size the comparison is directional rather than statistically decisive (Fisher exact p\approx0.37); a cross-backbone probe and public accumulating-context baselines are reported as operational comparisons rather than controlled tests of the contract variable itself. We release a reproducible testbed: 298 completed trajectories with condition tags, frozen memory/skill snapshots, prompt records, and analysis scripts -- an agent design and a validated, reusable methodology for studying how explicit memory layers shape long-horizon LLM-agent decisions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。