arXiv:2601.04786cs.LGcs.AI2026-01ACL被引 21

用图像压缩代理历史,大幅降低显存和算力消耗。

AgentOCR: Reimagining Agent History via Optical Self-Compression

  • 将对话历史转为图像,用视觉令牌高效存储
  • 减少超50%的token消耗,性能损失低于5%
  • 适合需要长期记忆的智能体部署场景

大语言模型驱动的智能体系统在多轮交互中表现出色,但文本历史快速膨胀导致显存和计算成本剧增。本文提出AgentOCR框架,将累积的观察-动作历史转化为紧凑渲染图像,利用视觉令牌更高的信息密度实现高效存储。为提升多轮推理效率,引入分段光学缓存机制:将历史按哈希可识别分段并维护视觉缓存,避免重复渲染。进一步提出智能体自压缩机制,让智能体主动控制压缩率,并通过感知压缩的奖励函数进行训练,动态平衡任务成功率与令牌效率。在ALFWorld和基于搜索的问答等挑战性基准上验证,AgentOCR在保持超过95%原始性能的同时,实现超过50%的令牌消耗降低,内存与计算效率显著提升。分析显示,光学缓存带来20倍渲染速度提升,自压缩机制有效协调效率与性能。代码已开源。

原文摘要 · Abstract (English)

Recent advances in large language models (LLMs) enable agentic systems trained with reinforcement learning (RL) over multi-turn interaction, but practical deployment is bottlenecked by rapidly growing textual histories that inflate token and memory costs. We introduce AgentOCR, a framework that exploits visual tokens' superior information density by representing the accumulated observation-action history as a compact rendered image. To make multi-turn rollouts scalable, AgentOCR proposes segment optical caching. By decomposing history into hashable segments and maintaining a visual cache, this mechanism eliminates redundant re-rendering. Beyond fixed rendering, AgentOCR introduces agentic self-compression, where the agent actively emits a compression rate and is trained with compression-aware reward to adaptively balance task success and token efficiency. We conduct extensive experiments on challenging agentic benchmarks, ALFWorld and search-based QA. Remarkably, AgentOCR preserves over 95\% of text-based agent performance while substantially reducing token consumption (>50\%), yielding consistent token and memory efficiency. Further analysis validates a 20$\times$ rendering speedup from optical caching and effective self-compression balancing. Our code is available at https://github.com/langfengQ/AgentOCR.

智能体视觉压缩内存优化强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。