用结构化结论链压缩多步问答的上下文，大幅降低推理成本。
ConCise: Training-Free Conclusion-Chain State Compression for Cost-Efficient Multi-Step RAG Services
- 用结论链替代原始文本累积，将上下文增长从O(N²)降至O(N)
- 单次调用生成推理与结论，节省64.63%平均令牌数
- 无需训练、兼容无服务器和边缘部署，即插即用
多步检索增强生成（RAG）广泛用于复杂问答的LLM服务中，通过迭代检索与推理实现高多跳准确率。但该范式导致历史文档和推理痕迹随轮次累积，使累计输入令牌数约呈$O(N^2)$增长，噪声密度持续上升。在API架构下，这直接放大了每请求的计费成本、网络负载和响应延迟。现有压缩方法依赖预训练模块或GPU级键值缓存访问，引入模型托管开销，不适用于API原生、无服务器及边缘部署。本文提出ConCise，一种无需训练的状态层协议，重构多步RAG服务中的跨轮上下文传输机制。具体地，ConCise将原始文本累积替换为追加式结构化结论链，使累积上下文增长从$O(N^2)$降至约$O(N)$。此外，引入融合生成机制，在单次API调用中联合输出推理与结论，消除串行双调用带来的重复输入计费。十二组配对实验覆盖三种模型、两个数据集和两种代表性框架，结果表明ConCise平均节省64.63%令牌，同时保持可接受的准确率，为低成本多步RAG服务优化提供即插即用、部署友好的解决方案。
原文摘要 · Abstract (English)
Multi-step retrieval-augmented generation (RAG) has been widely deployed as LLM-powered web services for complex question answering, where iterative retrieval-reasoning rounds deliver strong multi-hop accuracy. However, this paradigm causes historical documents and reasoning traces to accumulate across rounds, inflating cumulative input tokens approximately as $O(N^2)$ with progressively increasing noise density. In API-based service architectures, such growth directly amplifies per-request billing cost, network payload, and response latency. Existing compression approaches rely on pretrained modules or GPU-level KV cache access, introducing model hosting overhead incompatible with API-native, Serverless, and edge-side deployments. To address this issue, this paper proposes ConCise, a training-free state-layer protocol that restructures cross-round context transmission for multi-step RAG services. Specifically, ConCise replaces raw-text accumulation with an append-only chain of structured conclusions, compressing cumulative context growth from $O(N^2)$ to approximately $O(N)$. Furthermore, a fused generation mechanism is introduced to jointly emit reasoning and conclusions in a single API call, eliminating repeated input billing from serial dual-invocation overhead. Extensive experiments across twelve paired configurations spanning three models, two datasets, and two representative frameworks demonstrate that ConCise achieves 64.63\% average token savings while maintaining acceptable accuracy, providing a plug-and-play, deployment-friendly solution for cost-efficient multi-step RAG service optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。