解决零售搜索中跨区域数据异构问题，实现隐私保护下的个性化模型优化。
RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments
- 基于梯度层面的联邦学习框架，避免参数级方法在Transformer上的崩溃
- 在T5等模型上达到92.27%准确率，接近中心化最优结果
- 无需修改代码即可适配多种模型，适合跨区域个性化场景
零售搜索系统服务于具有不同查询模式、词汇和产品偏好的地理区域，导致显著的数据异构性，给隐私保护训练和模型个性化带来挑战。联邦学习提供了天然的隐私解决方案，但标准FL方法生成的全局模型会牺牲区域性能；现有个性化FL方法在参数层面操作，在现代Transformer模型（如T5）上因嵌入共享和层归一化交互而彻底失效（准确率低于10%）。我们提出RegionFed，一种架构鲁棒的联邦学习框架，完全在梯度层面运行，利用区域与全局梯度间的ℓ₂冲突作为统一信号，实现：(i) 异构性诊断，(ii) 自动选择最经济有效的个性化策略，(iii) 自适应控制个性化强度。由于将模型视为可微黑箱，RegionFed可零代码部署于T5-Small、T5-3B、RoBERTa和CNN，对Transformer带来显著提升（参数级方法已崩溃），并在CNN上保持稳定改进。在三个公开数据集（Amazon ESCI、Amazon Reviews、LEAF-FEMNIST）和四种架构上，RegionFed-Meta达到92.27%准确率，逼近隐私违规的中心化上界（92.04%，Δ=0.23pp，位于1σ内），同时提供(ε≈0.60)-差分隐私和O(1/√T)收敛速度。
原文摘要 · Abstract (English)
Retail search systems serve diverse geographic regions with distinct query patterns, vocabularies, and product preferences, creating significant data heterogeneity that challenges both privacy-preserving training and model personalization. Federated learning offers a natural solution for privacy, but standard FL methods produce global models that sacrifice regional performance, while existing personalized FL approaches operate at the parameter level and catastrophically collapse on modern transformers (below 10\% accuracy on T5) due to tied embeddings and LayerNorm interactions. We introduce RegionFed, an \textit{architecture-robust} federated learning framework that sidesteps this failure by operating entirely at the gradient level. RegionFed uses the $\ell_2$ conflict between regional and global gradients as a unified signal that (i) diagnoses heterogeneity, (ii) routes each region to the cheapest sufficient personalization strategy, and (iii) adaptively controls personalization strength. Because it treats models as differentiable black boxes, RegionFed deploys on T5-Small, T5-3B, RoBERTa, and CNN with zero code changes, providing large gains on transformers (where parameter-level methods collapse) and consistent improvements on CNNs. Across three public datasets (Amazon ESCI, Amazon Reviews, LEAF-FEMNIST) and four architectures, RegionFed-Meta achieves 92.27\%, closing the gap to the privacy-violating centralized upper bound (Centralized + Regional Weighting: 92.04\%, $\Delta$=0.23pp, within 1$\sigma$) while providing $(\epsilon{\approx}0.60)$-differential privacy and $\mathcal{O}(1/\sqrt{T})$ convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。