arXiv:2505.05031cs.IR2025-05KDD被引 8

让云端大模型与设备小模型协作,更好保护隐私并提升个性化回答质量。

LSRP: A Leader-Subordinate Retrieval Framework for Privacy-Preserving Cloud-Device Collaboration

  • 云端大模型通过用户间检索增强生成动态选择指导策略。
  • 设备小模型反馈优化使云端模型更贴合用户个人数据上下文。
  • 适合关注隐私保护与个性化推理的系统设计者使用。

云-设备协同利用云端大语言模型(LLMs)处理公共用户查询,设备端小语言模型(SLMs)处理私有用户数据,形成强大且隐私保护的解决方案。然而,现有方法常未能充分挖掘云端LLM的可扩展问题求解能力,同时低估了设备端SLM在访问和处理个性化数据方面的优势,导致两大问题:1)云端LLM的问题求解能力未与个性化用户任务需求对齐;2)用户数据未能有效融入设备端SLMs的响应,造成上下文信息不匹配。本文提出隐私保护云-设备协同的领导-从属检索框架(LSRP),通过:1)动态选择任务特定的领导策略——用户到用户检索增强生成(U-U-RAG),增强云端LLM对设备端SLM的指导;2)通过小模型反馈直接偏好优化(SMFB-DPO),将设备端SLM的数据优势融入云端模型,实现对齐。在两个数据集上的实验表明,LSRP持续优于现有最佳基线,显著提升问答相关性与个性化水平,同时通过高效的设备端检索保障用户隐私。代码已开源:https://github.com/Applied-Machine-Learning-Lab/LSRP。

原文摘要 · Abstract (English)

Cloud-device collaboration leverages on-cloud Large Language Models (LLMs) for handling public user queries and on-device Small Language Models (SLMs) for processing private user data, collectively forming a powerful and privacy-preserving solution. However, existing approaches often fail to fully leverage the scalable problem-solving capabilities of on-cloud LLMs while underutilizing the advantage of on-device SLMs in accessing and processing personalized data. This leads to two interconnected issues: 1) Limited utilization of the problem-solving capabilities of on-cloud LLMs, which fail to align with personalized user-task needs, and 2) Inadequate integration of user data into on-device SLM responses, resulting in mismatches in contextual user information. In this paper, we propose a Leader-Subordinate Retrieval framework for Privacy-preserving cloud-device collaboration (LSRP), a novel solution that bridges these gaps by: 1) enhancing on-cloud LLM guidance to on-device SLM through a dynamic selection of task-specific leader strategies named as user-to-user retrieval-augmented generation (U-U-RAG), and 2) integrating the data advantages of on-device SLMs through small model feedback Direct Preference Optimization (SMFB-DPO) for aligning the on-cloud LLM with the on-device SLM. Experiments on two datasets demonstrate that LSRP consistently outperforms state-of-the-art baselines, significantly improving question-answer relevance and personalization, while preserving user privacy through efficient on-device retrieval. Our code is available at: https://github.com/Applied-Machine-Learning-Lab/LSRP.

隐私计算云设备协同小模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。