提出高效加密下文本生成算法，显著降低隐私计算延迟
Efficient Decoding Methods for Language Models on Encrypted Data
- 设计cutmax算法，实现加密环境下的快速贪心解码
- 首次实现加密兼容的nucleus采样，速度比基线快24-35倍
- 支持梯度优化，适合需要隐私保护的生成任务
大型语言模型（LLMs）驱动现代AI应用，但在不可信服务器上处理敏感数据存在隐私风险。同态加密（HE）可实现加密数据上的安全推理，但神经文本生成所需的argmax和采样等解码方法为非多项式，加密环境下计算开销大，成为性能瓶颈。本文提出cutmax——一种面向HE的argmax算法，相比先前方法减少密文操作，实现高效贪婪解码。同时提出首个兼容HE的nucleus（top-p）采样方法，利用cutmax实现高效的随机解码，并具备可证明的隐私保障。两种技术均为多项式复杂度，支持隐私保护环境下的高效推理。此外，其可微性使梯度引导的序列级优化成为可能，替代传统的直通估计器。我们为cutmax提供强理论保证，通过指数放大最大值与次大值之差，证明其收敛性。在真实LLM输出上的评估显示，相较基线延迟降低24至35倍，推动了安全文本生成的发展。
原文摘要 · Abstract (English)
Large language models (LLMs) power modern AI applications, but processing sensitive data on untrusted servers raises privacy concerns. Homomorphic encryption (HE) enables computation on encrypted data for secure inference. However, neural text generation requires decoding methods like argmax and sampling, which are non-polynomial and thus computationally expensive under encryption, creating a significant performance bottleneck. We introduce cutmax, an HE-friendly argmax algorithm that reduces ciphertext operations compared to prior methods, enabling practical greedy decoding under encryption. We also propose the first HE-compatible nucleus (top-p) sampling method, leveraging cutmax for efficient stochastic decoding with provable privacy guarantees. Both techniques are polynomial, supporting efficient inference in privacy-preserving settings. Moreover, their differentiability facilitates gradient-based sequence-level optimization as a polynomial alternative to straight-through estimators. We further provide strong theoretical guarantees for cutmax, proving its convergence via exponential amplification of the gap ratio between the maximum and runner-up elements. Evaluations on realistic LLM outputs show latency reductions of 24x-35x over baselines, advancing secure text generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。