arXiv:2510.03215cs.CLcs.LG2025-10被引 44

让大模型直接交换内部记忆,提速增效还更准。

Cache-to-Cache: Direct Semantic Communication Between Large Language Models

  • 用神经网络融合两模型的键值缓存,实现语义直传。
  • 准确率比单模型高6.4%-14.2%,比文本通信高3.1%-5.4%。
  • 适合多模型协同系统,尤其看重速度与精度的场景。

多大模型系统利用不同大模型的互补优势,在性能和效率上超越单个模型。现有设计中,模型间通过文本通信,迫使内部表示转换为输出标记序列,导致丰富语义信息丢失,并产生逐标记生成延迟。受此启发,我们提出:大模型能否超越文本通信?基于理想实验,丰富键值缓存(KV-Cache)语义可提升响应质量而无需增加缓存大小,验证了其作为跨模型通信介质的有效性。因此,我们提出缓存到缓存(Cache-to-Cache, C2C)新范式,通过神经网络将源模型的KV缓存与目标模型的缓存投影融合,实现直接语义传递。可学习门控机制选择受益于缓存通信的目标层。相比文本通信,C2C充分利用双模型深层专业化语义,避免显式中间文本生成。实验表明,C2C平均准确率高于单模型6.4%-14.2%;相较于文本通信提升3.1%-5.4%,同时平均延迟降低2.5倍。代码已开源:https://github.com/thu-nics/C2C。

原文摘要 · Abstract (English)

Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5x speedup in latency. Our code is available at https://github.com/thu-nics/C2C.

大模型协同语义通信缓存优化推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。