对比八种记忆压缩策略，发现其对科学发现任务效果相似但成本差异大。
Evaluating Memory Condensation Strategies for Coding Agents in Data-Driven Scientific Discovery
- 用GPT-4o在60个任务上测试8种记忆压缩方法
- 基于LLM的压缩使令牌开销增加24%-94%，但不提升假设质量
- 屏蔽工具调用输出可节省8.6%资源，适合数据驱动科研
编码代理在长时间任务中积累大量上下文，但固定上下文窗口迫使用户在截断和任务失败间权衡。尽管已有从滑动窗口到大模型生成摘要等多种记忆压缩策略，但尚无系统性比较来指导科学发现任务中的策略选择。本文使用GPT-4o在涵盖六个科学领域的60个DiscoveryBench任务（共480次评估）上评估了八种记忆压缩策略。结果表明，不同压缩器对假设质量无显著影响；而基于LLM的压缩方式使令牌成本增加24%-94%；屏蔽工具调用输出可实现8.6%的净节省。此外，数据驱动科学发现任务的最优压缩器因学科领域和任务时长而异。
原文摘要 · Abstract (English)
Coding agents accumulate extensive context during long-running tasks, yet fixed context windows force practitioners to choose between truncation and task failure. While numerous memory condensation strategies have been proposed, from simple sliding windows to LLM-generated summaries, no systematic comparison exists to guide strategy selection, especially in scientific discovery tasks. We evaluate eight memory condensation strategies using GPT-4o on sixty DiscoveryBench tasks spanning six scientific domains (480 total evaluations). We find that no condenser significantly alters hypothesis quality, while LLM-based condensers increase token costs by 24-94 percent, and masking tool-call outputs achieves an 8.6 percent net savings. We also observe that the optimal condenser for data-driven scientific discovery varies by scientific domain and task length.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。