给大模型加个索引,能大幅降低推理时的查找成本。
The Library Theorem: How External Organization Governs Agentic Reasoning Capacity
- 用索引结构替代逐条扫描,实现对记忆内容的快速定位。
- 索引使查找成本从线性增长降为对数增长,规模越大越明显。
- 适合需要高效推理的智能体系统,尤其在长链条决策中。
外部推理已通过思维链被基于Transformer的智能体利用,但结构化检索——即对自身推理状态进行索引——仍未充分探索。我们将Transformer上下文窗口形式化为输入/输出页,并证明:配备索引外部记忆的工具增强型智能体,其每次查询的检索成本为 $O(\log_b N)$ 页读取,远低于仅支持顺序扫描者的 $Ω(N)$;在 $T$ 步推理中累计成本从 $Θ(T^2)$ 降至 $O(T \log_b T)$,差距随思辨深度扩大。我们在三个内容类型(随机哈希、有序整数、百科条目)上测试该预测,存储量从50到5,000项不等,涵盖GPT-4o-mini与GPT-5.4两个模型版本。在抽象内容上,索引智能体无论存储规模如何,中位数仅需1次页读取,验证了 $O(1)$ 预期。无索引的有序页面无法弥合差距:弱模型无法在大规模下维持二分查找,强模型虽接近 $\log_2 N$ 但仍比索引慢5倍。在熟悉内容(百科条目)中,另一失效模式出现:模型识别领域后绕过检索协议,直接从参数记忆生成答案,造成灾难性令牌消耗。这揭示出索引所整合的两种认知操作——内容理解(语言模型优势)与导航协议遵循(模型易失误)——应分离:用语言模型构建索引,用确定性算法遍历索引。
原文摘要 · Abstract (English)
Externalized reasoning is already exploited by transformer-based agents through chain-of-thought, but structured retrieval -- indexing over one's own reasoning state -- remains underexplored. We formalize the transformer context window as an I/O page and prove that tool-augmented agents with indexed external memory achieve exponentially lower retrieval cost than agents restricted to sequential scanning: $O(\log_b N)$ versus $Ω(N)$ page reads per query, and $O(T \log_b T)$ versus $Θ(T^2)$ cumulative cost over $T$ reasoning steps -- a gap that widens as deliberation deepens. We test these predictions on a controlled lookup benchmark across three content types -- random hashes, ordered integers, and encyclopedia entries -- varying store size from 50 to 5,000 items, and replicate key conditions across two model generations (GPT-4o-mini and GPT-5.4). On abstract content, the indexed agent achieves median 1 page read regardless of store size, confirming the $O(1)$ prediction. Sorted pages without an index fail to close the gap: the weaker model cannot sustain binary search at scale, and the stronger model achieves near-optimal $\log_2 N$ search but still loses to the index by $5\times$. On familiar content (encyclopedia entries), a competing failure mode emerges: the model recognizes the domain, bypasses the retrieval protocol, and generates answers from parametric memory, producing catastrophic token expenditure even when the index is sound. This parametric memory competition dissociates the two cognitive operations that indexing combines: understanding content (where language models excel) and following navigational protocols (where they fail when understanding tempts them to shortcut). The result argues for a separation of concerns: use language models for index construction, where semantic understanding helps, and deterministic algorithms for index traversal, where it hurts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。