用视觉编码存储长序列记忆,高效检索不丢失细节
OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory

- 将历史轨迹转为带标识的图像,以视觉方式高密度存储
- 通过定位+抄写机制精准还原原文,减少幻觉
- 适合需要长期记忆的交互式智能体,尤其在上下文受限时
自主大模型智能体在长周期、交互式环境中表现越来越重要,成功依赖于对长期经验的复用。然而现有记忆系统受限于文本上下文长度:存储原始轨迹代价过高,而摘要与纯文本检索会损失信息并导致证据碎片化。为此,我们提出光学上下文检索记忆(OCR-Memory),利用视觉模态作为经验的高密度表示,实现任意长度历史的保留,且检索时提示开销极低。具体地,OCR-Memory将历史轨迹渲染为带有唯一视觉标识的图像。通过“定位-抄写”范式,基于视觉锚点选择相关区域,直接提取原始文本,避免自由生成,降低幻觉风险。在长周期智能体基准测试中,即使在严格上下文限制下仍持续取得提升,证明光学编码能显著增强有效记忆容量,并保证证据恢复的准确性。
原文摘要 · Abstract (English)
Autonomous LLM agents increasingly operate in long-horizon, interactive settings where success depends on reusing experience accumulated over extended histories. However, existing agent memory systems are fundamentally constrained by text-context budgets: storing or revisiting raw trajectories is prohibitively token-expensive, while summarization and text-only retrieval trade token savings for information loss and fragmented evidence. To address this limitation, we propose Optical Context Retrieval Memory (OCR-Memory), a memory framework that leverages the visual modality as a high-density representation of agent experience, enabling retention of arbitrarily long histories with minimal prompt overhead at retrieval time. Specifically, OCR-Memory renders historical trajectories into images annotated with unique visual identifiers. OCR-Memory retrieves stored experience via a \emph{locate-and-transcribe} paradigm that selects relevant regions through visual anchors and retrieves the corresponding verbatim text, avoiding free-form generation and reducing hallucination. Experiments on long-horizon agent benchmarks show consistent gains under strict context limits, demonstrating that optical encoding increases effective memory capacity while preserving faithful evidence recovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。