用Git管理大模型推理过程,实现可追溯、可合并的思维记忆。
GitOfThoughts: Version-Controlled Reasoning and Agent Memory You Can Replay, Diff, and Merge

- 将推理过程转为Git仓库,每步思考变提交,结果变标签
- 实验显示记忆仅在问题高度相似时有效(余弦相似度>0.8)
- 优势在于可审计、可回溯、支持多智能体记忆合并
大型语言模型的推理过程一旦完成便不留痕迹。当上下文窗口关闭后,链式思考步骤消失,被剪枝的搜索分支无法恢复,而内存缓冲区也无法进行差异比对、合并或审计。代码、基础设施和实验均已实现版本控制,唯独推理未被记录。GitOfThoughts将智能体的推理树存储为Git仓库:每个带评分的思考成为一次提交,评分作为注释,结果变为标签,检索则通过git log查询自身历史实现。我们测试了一个简单问题:给智能体加入过往问题的记忆是否能提升准确性?在两个基准测试、两种模型规模及多次预注册重复实验中,对比了五种记忆存储方式(无记忆、Markdown文件、向量数据库、图谱、Git)。结果显示,在新问题上,记忆并未带来提升,包括一个早期看似有希望的结果在重复实验中也未能成立。只有当问题与记忆中的内容高度相似(余弦相似度超过0.8)时,记忆才起作用;低于此阈值则无效。这表明模型是在寻找答案而非学习方法。即使模型规模扩大4.5倍,也无法从已解决示例中提取出可复用的方法,仅更擅长识别近似匹配。唯一对新问题始终有效的策略是生成多个答案并选择最常见者(自一致性)。因此,使用Git作为记忆存储的优势不在于检索性能,而在于可审计性、历史追踪能力以及支持多智能体记忆合并,且不影响准确性。
原文摘要 · Abstract (English)
Large language model reasoning leaves no trace once it is done. The steps of a chain of thought disappear when the context window closes, a pruned search branch is just gone, and memory buffers cannot be diffed, merged, or audited. Code, infrastructure, and experiments are all version-controlled. Reasoning is not. GitOfThoughts stores an agent's reasoning tree as a git repository. Every scored thought becomes a commit, scores become notes, outcomes become tags, and retrieval is just git log over the agent's own history. We use this to test something simple. Does giving an agent memory from past problems actually make it more accurate? We tried five memory stores (none, a markdown file, a vector database, a graph, and git) across two benchmarks, two model sizes, and several pre-registered repeat experiments. The answer, on new problems, is no, including one promising early result that did not hold up when we repeated it. Memory only helps once the problem being solved is nearly identical to something already in memory (cosine similarity above about 0.8); below that, it does nothing. In other words, the model is finding the answer rather than learning the method. Even a model 4.5x larger still cannot pull a reusable method out of a worked example; it just gets better at spotting near-copies. The only thing that reliably helped on new problems was generating several answers and picking the most common one (self-consistency). So the case for using git as the memory store is not that it retrieves better. It is that it gives auditability, history, and the ability to merge two agents' memories, at no cost to accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。