用视觉布局智能压缩记忆，让模型在有限上下文里更高效推理
MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoning
- 将文本记忆转为图像，按信息重要性动态分配存储空间
- 在极端上下文限制下，多跳问答任务性能超越纯文本基线
- 适合需要长期推理的复杂任务，如长文档问答与规划
长时序代理推理需要将不断增长的交互历史有效压缩到有限的上下文窗口中。现有记忆系统通常将历史序列化为文本，每个词元成本均等且随长度线性增长，常在低价值细节上浪费宝贵预算。为此，我们提出MemOCR，一种基于视觉布局的多模态记忆代理，通过自适应信息密度分配内存空间，提升在严格上下文预算下的长时序推理能力。具体而言，MemOCR维护结构化富文本记忆（如标题、高亮），并将其渲染为图像供代理查阅，视觉上优先呈现关键证据，同时大幅压缩次要细节。为确保在不同内存预算下的鲁棒性，我们在预算感知目标下使用强化学习训练MemOCR，使其接触多种压缩程度。在多个长上下文多跳与单跳问答基准测试中，MemOCR优于强文本基线，并在极端预算下实现更高效的上下文利用。
原文摘要 · Abstract (English)
Long-horizon agentic reasoning necessitates effectively compressing growing interaction histories into a limited context window. Most existing memory systems serialize history as text, where token-level cost is uniform and scales linearly with length, often spending scarce budget on low-value details. To this end, we introduce MemOCR, a multimodal memory agent that improves long-horizon reasoning under tight context budgets by allocating memory space with adaptive information density through visual layout. Concretely, MemOCR maintains a structured rich-text memory (e.g., headings, highlights) and renders it into an image that the agent consults for memory access, visually prioritizing crucial evidence while aggressively compressing auxiliary details. To ensure robustness across varying memory budgets, we train MemOCR with reinforcement learning under budget-aware objectives that expose the agent to diverse compression levels. Across long-context multi-hop and single-hop question-answering benchmarks, MemOCR outperforms strong text-based baselines and achieves more effective context utilization under extreme budgets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。