解决大模型工具输出溢出问题,实现无损长文本处理
Solving Context Window Overflow in AI Agents
- 用记忆指针替代原始数据,突破上下文窗口限制
- 实测减少约七倍的令牌消耗,任务成功率不变
- 适合需要完整工具输出的科研类智能体场景
大型语言模型(LLMs)已具备与外部工具交互的能力,可获取训练数据之外的专业知识,这在化学和材料科学等动态、知识密集型领域至关重要。然而,大型工具输出常超出LLM的上下文窗口,导致任务无法完成。现有方法如截断或摘要会丢失完整信息,不适用于需全量数据的流程。本文提出一种新方法,使LLM能无损处理任意长度的工具响应。通过将模型交互从原始数据转向记忆指针,该方法保持了工具功能,支持无缝集成到智能体工作流中,并显著降低令牌使用量和执行时间。在真实材料科学应用中验证,传统工作流无法执行的任务,本方法成功完成。对比实验显示,该方法仅消耗传统流程约七分之一的令牌。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have become increasingly capable of interacting with external tools, granting access to specialized knowledge beyond their training data - critical in dynamic, knowledge-intensive domains such as Chemistry and Materials Science. However, large tool outputs can overflow the LLMs' context window, preventing task completion. Existing solutions such as truncation or summarization fail to preserve complete outputs, making them unsuitable for workflows requiring the full data. This work introduces a method that enables LLMs to process and utilize tool responses of arbitrary length without loss of information. By shifting the model's interaction from raw data to memory pointers, the method preserves tool functionality, allows seamless integration into agentic workflows, and reduces token usage and execution time. The proposed method is validated on a real-world Materials Science application that cannot be executed with conventional workflows, and its effectiveness is demonstrated via a comparative analysis where both methods succeed. In this experiment, the proposed approach consumed approximately seven times fewer tokens than the traditional workflow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。