用强化学习优化提示重建,显著降低隐私泄露攻击成本。
OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi-tenant LLM Services
- 通过强化学习与两阶段微调,提升提示重构效率。
- 在3B到14B模型上,平均请求次数减少12.48倍。
- 自动识别敏感词,避免人工标注和过拟合问题。
多租户大模型服务广泛采用共享键值缓存以提高效率,但由此产生的侧信道漏洞可能引发提示泄露攻击。以往研究虽揭示了此类攻击面,但主要关注扩大攻击向量,而非优化攻击性能,报告的攻击成本过高,低估了真实隐私风险。本文提出OptiLeak,一种基于强化学习的框架,通过两阶段微调最大化提示重建效率。核心思路是利用似然排序自动识别领域特异的“难预测词”(hard tokens),这些词难以预测却携带敏感信息,进而构建偏好对用于直接偏好优化,避免了人工标注并缓解了扩展监督微调中的过拟合问题。在涵盖医疗与金融领域的三个基准上评估,OptiLeak相较基线方法平均每次令牌请求次数减少高达12.48倍,且在3B至14B参数规模的模型上均表现稳定提升。结果表明,基于缓存的提示泄露威胁远比此前估计严重,凸显生产环境中实施强缓存隔离的必要性。
原文摘要 · Abstract (English)
Multi-tenant LLM serving frameworks widely adopt shared Key-Value caches to enhance efficiency. However, this creates side-channel vulnerabilities enabling prompt leakage attacks. Prior studies identified these attack surfaces yet focused on expanding attack vectors rather than optimizing attack performance, reporting impractically high attack costs that underestimate the true privacy risk. We propose OptiLeak, a reinforcement learning-enhanced framework that maximizes prompt reconstruction efficiency through two-stage fine-tuning. Our key insight is that domain-specific ``hard tokens'' -- terms difficult to predict yet carrying sensitive information -- can be automatically identified via likelihood ranking and used to construct preference pairs for Direct Preference Optimization, eliminating manual annotation. This enables effective preference alignment while avoiding the overfitting issues of extended supervised fine-tuning. Evaluated on three benchmarks spanning medical and financial domains, OptiLeak achieves up to $12.48\times$ reduction in average requests per token compared to baseline approaches, with consistent improvements across model scales from 3B to 14B parameters. Our findings demonstrate that cache-based prompt leakage poses a more severe threat than previously reported, underscoring the need for robust cache isolation in production deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。