保护用户隐私的LLM路由框架，实现高效安全的模型选择。
Privacy-Preserving LLMs Routing
- 采用多方计算友好操作加速编码器推理。
- 在加密环境下保持路由质量，性能接近明文水平。
- 提出低通信开销的Top-k算法，显著降低延迟。
大型语言模型（LLM）路由已成为平衡模型性能与成本效率的关键策略，通过动态选择不同模型提供商的服务实现。然而，LLM路由在用户与模型之间引入中间层，带来新的用户数据隐私风险，而此类风险尚未被系统研究。尽管密码学技术如安全多方计算（MPC）可实现隐私保护计算，但其协议设计与实现仍不成熟，直接应用通常导致高昂的计算开销。为此，我们提出隐私保护的LLM路由框架PPRoute。PPRoute包含多项优化策略，以在MPC下加速编码器推理和最近邻搜索，并维持路由质量。首先，采用MPC友好的操作提升编码器推理速度；其次，提出多阶段模型训练算法，在加密域约束下仍保持路由精度；第三，设计一种无序Top-k算法，通信复杂度为O(1)，显著减少模型搜索中的通信延迟。在多个数据集上，PPRoute达到与明文方法相当的性能，且相较朴素MPC实现提速约20倍。
原文摘要 · Abstract (English)
Large language model (LLM) routing has emerged as a critical strategy to balance model performance and cost-efficiency by dynamically selecting services from various model providers. However, LLM routing adds an intermediate layer between users and LLMs, creating new privacy risks to user data. These privacy risks have not been systematically studied. Although cryptographic techniques such as Secure Multi-Party Computation (MPC) enable privacy-preserving computation, their protocol design and implementation remain under-explored, and naïve implementations typically incur prohibitive computational overhead. To address this, we propose a privacy-preserving LLM routing framework (PPRoute). PPRoute includes multiple strategies to speed up encoder inference and nearest neighbor search under the MPC and maintain the quality of LLM routing. First, PPRoute uses MPC-friendly operations to boost the encoder inference. Second, PPRoute uses a multiple-step model training algorithm to maintain routing quality despite the constraints of the encrypted domain. Third, PPRoute proposes an unsorted Top-k algorithm with $O(1)$ communication complexity for secure sorting in model search, significantly reducing communication latency. Across different datasets, PPRoute achieves the performance of plaintext counterparts, while achieving approximately a 20$\times$ speedup over naïve MPC implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。