让模型在切换任务时仍能记住更新后的映射关系，靠的是可学习的内存整合机制。
Consolidator: Learning Persistent Routed Memory Across Context Boundaries
- 设计共享槽位算子Consolidator，在不重放原始数据情况下将短期记忆转为长期记忆。
- 经两次整合后，对更新映射的召回率从44.38%提升至87.02%，而短期记忆保持在89.90%。
- 适合研究记忆持久性与访问控制协同的场景，如多阶段推理任务。
将短期记忆（STM）复制到更慢的存储中可实现状态跨上下文边界保留，但持久性本身并不保证该状态会影响后续记忆访问。我们在相位记忆网络（PMNet）中通过Consolidator测试这一区别，该共享槽位局部算子在路由后转换STM并累积至长期记忆（LTM），无需重放源标记。每次整合后清除键值缓存和STM。保留的LTM仍可读取，并输入层级路由器，从而决定后续输入访问哪些显式记忆槽。我们在一个两段式模10映射任务上评估此机制，第二段在同一内存地址更新映射。第二次整合与重置后，未见样本查询需从LTM恢复更新后的映射。主干与记忆接口冻结，仅12.35K个Consolidator参数可训练（占29.95M模型的0.041%）。五组来自同一STM预训练检查点的配对运行显示，直接使用LTM路由使更新映射召回率从44.38±1.94%提升至87.02±1.76%（+42.64±1.10个百分点），而即时STM召回率在两种条件下均为89.90%；两者分别训练Consolidator且保留相同的LTM读路径。学习到的整合方式在无路由时优于强制恒等累积21.40±1.91个百分点，在有路由时则高出68.70±1.76个百分点。因此，在该任务中，整合后的LTM既作为可检索内容，也作为影响后续槽选择的访问状态。
原文摘要 · Abstract (English)
Copying short-term memory (STM) into a slower store can preserve state across a context boundary, but persistence alone does not ensure that the retained state influences subsequent memory access. We test this distinction in a Phasor Memory Network (PMNet) using Consolidator, a shared slot-local operator that transforms routed STM before accumulating it into long-term memory (LTM), without replaying the source tokens. After each consolidation, the KV cache and STM are cleared. The retained LTM can still be read and is also fed into the hierarchical router, thereby conditioning which explicit-memory slots subsequent inputs access. We evaluate this mechanism on a two-segment modulo-10 mapping task in which the second segment updates the mapping at the same memory address. Following a second consolidation and reset, a held-out query must recover the updated mapping from LTM. The backbone and memory interface are frozen, leaving only 12.35K Consolidator parameters trainable (0.041\% of a 29.95M model). Across five paired runs from the same STM-pretraining checkpoint, direct LTM routing raises updated-mapping recall from $44.38\pm1.94\%$ to $87.02\pm1.76\%$ ($+42.64\pm1.10$ percentage points), while immediate STM recall remains 89.90\% in both conditions; both train separate Consolidators and retain the same LTM read paths. Learned consolidation outperforms forced identity accumulation by $21.40\pm1.91$ percentage points without routing and $68.70\pm1.76$ with routing. Thus, on this task, consolidated LTM serves as both retrievable content and an access state that shapes subsequent slot selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。