根据输入动态选模型,让加密推理更快更省。
SecureRouter: Encrypted Routing for Efficient Secure Inference

- 加密状态下动态选择不同大小的模型,适应输入需求。
- 相比以往方法,延迟降低1.95倍,精度几乎不变。
- 适合需要高效安全推理的云服务与隐私保护场景。
基于密码学的神经网络推理通常依赖安全计算技术(如多方计算,MPC),使云服务器在不解密客户端输入的情况下进行处理。尽管已有隐私保护推理系统将网络优化与MPC协同设计,但其仍存在速度慢、成本高的问题,限制了实际部署。主要瓶颈在于对所有加密输入均使用单一固定Transformer模型,忽略了不同输入对模型规模的需求差异。本文提出SecureRouter,一个端到端的加密路由与推理框架,通过加密状态下的输入自适应模型选择来加速安全Transformer推理。该框架构建统一的加密流水线,集成安全路由模块与MPC优化的模型池,实现路由、推理与协议执行的协同,同时保证数据和模型的完全保密。系统包含训练阶段与推理阶段组件:一个考虑MPC开销的安全路由器,可从加密特征中预测各模型的效用与成本;以及一个架构与量化方案协同训练的MPC优化模型池,以最小化MPC通信与计算开销。相较于现有工作,SecureRouter在几乎无精度损失下实现1.95倍的延迟降低,为可扩展、高效的隐私保护AI推理提供了实用路径。开源代码已发布于:https://github.com/UCF-ML-Research/SecureRouter
原文摘要 · Abstract (English)
Cryptographically secure neural network inference typically relies on secure computing techniques such as Secure Multi-Party Computation (MPC), enabling cloud servers to process client inputs without decrypting them. Although prior privacy-preserving inference systems co-design network optimizations with MPC, they remain slow and costly, limiting real-world deployment. A major bottleneck is their use of a single, fixed transformer model for all encrypted inputs, ignoring that different inputs require different model sizes to balance efficiency and accuracy. We present SecureRouter, an end-to-end encrypted routing and inference framework that accelerates secure transformer inference through input-adaptive model selection under encryption. SecureRouter establishes a unified encrypted pipeline that integrates a secure router with an MPC-optimized model pool, enabling coordinated routing, inference, and protocol execution while preserving full data and model confidentiality. The framework includes training-phase and inference-phase components: an MPC-cost-aware secure router that predicts per-model utility and cost from encrypted features, and an MPC-optimized model pool whose architectures and quantization schemes are co-trained to minimize MPC communication and computation overhead. Compared to prior work, SecureRouter achieves a latency reduction by 1.95x with negligible accuracy loss, offering a practical path toward scalable and efficient secure AI inference. Our open-source implementation is available at: https://github.com/UCF-ML-Research/SecureRouter
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。