让AI系统缓存推理过程而非仅结果，大幅减少重复计算。
SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systems
- 将生成流程拆分为意图解析与可视化合成，缓存中间结构化结果。
- 缓存命中率从38.7%提升至83.10%，节省4023次LLM调用。
- 适合构建高效智能代理系统的开发者参考。
智能代理系统存在隐藏的低效问题：即使用户提问语言全新，仍频繁重复相同的中间推理步骤，如指标归一化或图表框架搭建。传统边界缓存因将推理视为黑盒而无法捕捉此效率损失。我们提出SemanticALLI，一种在Alli（PMG营销智能平台）中实现的流水线感知架构，旨在显式缓存冗余推理过程。通过将生成分解为分析意图解析（AIR）和可视化合成（VS），SemanticALLI将结构化的中间表示（IRs）提升为可缓存的一等对象。评估显示，基线单体缓存因语言差异限制，命中率仅为38.7%；而我们的结构化方法引入可视化合成阶段后，命中率跃升至83.10%，绕过4023次LLM调用，中位延迟仅2.66毫秒。内部复用显著降低总令牌消耗，揭示了实用设计启示：即使用户几乎不重复提问，流水线本身却常在稳定、结构化的检查点上重复相同逻辑，此处正是缓存最可靠之处。
原文摘要 · Abstract (English)
Agentic AI pipelines suffer from a hidden inefficiency: they frequently reconstruct identical intermediate logic, such as metric normalization or chart scaffolding, even when the user's natural language phrasing is entirely novel. Conventional boundary caching fails to capture this inefficiency because it treats inference as a monolithic black box. We introduce SemanticALLI, a pipeline-aware architecture within Alli (PMG's marketing intelligence platform), designed to operationalize redundant reasoning. By decomposing generation into Analytic Intent Resolution (AIR) and Visualization Synthesis (VS), SemanticALLI elevates structured intermediate representations (IRs) to first-class, cacheable artifacts. The impact of caching within the agentic loop is substantial. In our evaluation, baseline monolithic caching caps at a 38.7% hit rate due to linguistic variance. In contrast, our structured approach allows for an additional stage, the Visualization Synthesis stage, to achieve an 83.10% hit rate, bypassing 4,023 LLM calls with a median latency of just 2.66 ms. This internal reuse reduces total token consumption, offering a practical lesson for AI system design: even when users rarely repeat themselves, the pipeline often does, at stable, structured checkpoints where caching is most reliable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。