提出安全共享机制，在不牺牲性能前提下防止大模型推理中的隐私泄露。
Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference
- 三阶段异步检测分离隐私判断与推理，支持流式处理。
- 采用路径压缩的基数树管理器，实现敏感数据的精准隔离。
- 通过重用多样性比监控残留泄漏，适合多租户部署场景。
全局键值缓存共享可有效加速大语言模型推理，但会引入可被攻击者利用的时序侧信道，导致跨租户隐私泄露。为此，我们提出SafeKV（安全灵活的键值缓存共享）——一种系统级隐私保护与缓存管理协同设计。SafeKV在服务运行时集成轻量级检测与隔离机制，消除威胁模型下的敏感缓存块跨租户复用，同时保留大部分全局共享性能优势。核心贡献包括：(1) 三层次异步检测流水线，将隐私分类与推理解耦，支持流式工作负载；(2) 基于基数树的统一内存管理器，结合路径压缩与敏感度感知淘汰策略，实现可扩展的选择性隔离；(3) 基于重用多样性比（RDR）的运行时防护机制，检测并限制残余信息泄露。在大型LLM后端上，相比完全隔离，SafeKV将首次令牌延迟（TTFT）开销降低最多40.58%，吞吐提升最高达2.66倍。总体而言，SafeKV在恢复缓存复用效率的同时，为多租户大模型推理提供了强且实用的隐私保障。
原文摘要 · Abstract (English)
Global KV-cache sharing is an effective optimization for accelerating large language model (LLM) inference, yet it introduces an API-visible timing side channel that lets adversaries infer sensitive user inputs from shared entries, leading to cross-tenant privacy risks. To address this problem, we introduce SafeKV (Secure and Flexible KV-cache Sharing), a system-level co-design of privacy enforcement and KV-cache management. SafeKV integrates lightweight detection and isolation directly into the serving runtime to eliminate cross-tenant reuse of sensitive KV-cache blocks under our threat model, while recovering most of the performance benefits of global sharing. Our key contributions are: (1) a three-tier asynchronous detection pipeline that decouples privacy classification from inference and supports streaming workloads, (2) a unified radix-tree-based memory manager with path compression and sensitivity-aware eviction for scalable selective isolation, and (3) an RDR-guided (Reuse Diversity Ratio) runtime safeguard that detects and bounds residual leakage. On large LLM backends, SafeKV reduces the time-to-first-token (TTFT) overhead compared to full isolation by up to 40.58% and raises throughput by up to 2.66x. Overall, SafeKV restores the efficiency of KV reuse while enforcing strong, practical privacy for multi-tenant LLM inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。