用私有模型替换数据，实现隐私保护的LLM定制服务
Model-based Large Language Model Customization as Service
- 客户上传领域模型而非原始数据，通过连接模块嵌入基础大模型
- 在相同隐私约束下，领域准确率比现有方法提升26%
- 无需提供上下文信息，推理效率与原服务相当，适合高隐私场景
主流大语言模型服务（如OpenAI、Google）在通用任务上表现优异，但在领域特定应用中常表现不佳。当前定制化服务通常需用户上传数据进行微调，带来严重隐私风险。虽然差分隐私（DP）数据合成可作为替代，但过度加噪导致效果低下。为此，我们提出Llamdex框架，支持将领域特定预训练模型作为服务进行定制，客户端上传模型（可选低噪声差分隐私保护），通过连接模块插入基础模型。关键的是，这些连接模块无需敏感数据即可训练，使客户在不泄露数据的前提下完成定制。实验表明，在相同隐私约束下，Llamdex相较最先进私有数据合成方法，领域准确率提升最高达26%；同时避免用户在查询中提供领域上下文，推理效率与原服务相当。
原文摘要 · Abstract (English)
Prominent Large Language Model (LLM) services from providers like OpenAI and Google excel at general tasks but often underperform on domain-specific applications. Current customization services for these LLMs typically require users to upload data for fine-tuning, posing significant privacy risks. While differentially private (DP) data synthesis presents a potential alternative, its application commonly results in low effectiveness due to the introduction of excessive noise on data for DP. To overcome this, we introduce Llamdex, a novel framework that facilitates LLM customization as a service, where the client uploads pre-trained domain-specific models rather than data. This client-uploaded model, optionally protected by DP with much lower noise, is inserted into the base LLM via connection modules. Significantly, these connecting modules are trained without requiring sensitive domain data, enabling clients to customize LLM services while preserving data privacy. Experiments demonstrate that Llamdex improves domain-specific accuracy by up to 26% over state-of-the-art private data synthesis methods under identical privacy constraints and, by obviating the need for users to provide domain context within queries, maintains inference efficiency comparable to the original LLM service.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。