轻量级优化推荐系统嵌入参数，提升联邦学习通信效率
Plug-and-Play Parameter-Efficient Tuning of Embeddings for Federated Recommendation
- 用可插拔的高效微调技术减少嵌入参数传输量
- 在多个数据集上降低通信开销同时保持精度
- 适合关注隐私保护与通信效率的推荐系统研究者
随着云边协同的发展，推荐服务越来越多地在分布式环境中训练。联邦推荐（FR）通过共享模型参数而非原始数据，实现多方协作训练并保护隐私。然而，由于物品嵌入参数数量庞大，严重制约了通信效率。现有研究多聚焦于模型效率优化，却忽视了嵌入参数的开销问题。为此，我们提出一种基于参数高效微调（PEFT）的嵌入优化框架，显著减少需传输的嵌入参数量。该方法为轻量级、可插拔设计，可无缝集成至现有联邦推荐方法中。除引入常见的LoRA和哈希编码外，还探索使用残差量化变分自编码器（RQ-VAE）作为新型PEFT策略。在多种FR模型架构与数据集上的实验表明，该框架有效降低通信开销并提升准确率。源代码已开源：https://github.com/young1010/FedPEFT。
原文摘要 · Abstract (English)
With the rise of cloud-edge collaboration, recommendation services are increasingly trained in distributed environments. Federated Recommendation (FR) enables such multi-end collaborative training while preserving privacy by sharing model parameters instead of raw data. However, the large number of parameters, primarily due to the massive item embeddings, significantly hampers communication efficiency. While existing studies mainly focus on improving the efficiency of FR models, they largely overlook the issue of embedding parameter overhead. To address this gap, we propose a FR training framework with Parameter-Efficient Fine-Tuning (PEFT) based embedding designed to reduce the volume of embedding parameters that need to be transmitted. Our approach offers a lightweight, plugin-style solution that can be seamlessly integrated into existing FR methods. In addition to incorporating common PEFT techniques such as LoRA and Hash-based encoding, we explore the use of Residual Quantized Variational Autoencoders (RQ-VAE) as a novel PEFT strategy within our framework. Extensive experiments across various FR model backbones and datasets demonstrate that our framework significantly reduces communication overhead while improving accuracy. The source code is available at https://github.com/young1010/FedPEFT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。