arXiv:2512.00841cs.LG2025-12

用预测空间知识交易,大幅降低多媒体联邦学习通信量

Prediction-space knowledge markets for communication-efficient federated learning on multimedia tasks

  • 客户端只传预测结果,不传模型参数
  • 在低通信开销下准确率超传统方法近10倍
  • 适合资源受限的移动端多媒体任务

联邦学习可在分布式多媒体数据上协同训练,但面临统计异质性和通信限制,尤其在客户端使用大模型时。经典参数平均方法如FedAvg在非独立同分布数据下易发散。本文提出KTA v2,一种基于预测空间的知识交易市场:每轮客户端本地训练后,仅共享在小公共参考集上的logits;服务器构建客户端间预测空间相似图,结合参考集准确率生成个性化教师模型,返回软目标用于第二阶段知识蒸馏更新。该两阶段过程可解释为统一目标下的近似块坐标下降,带有预测空间正则化。在FEMNIST、CIFAR-10和AG News上的实验表明,相比本地基线和强参数方法(FedAvg、FedProx),KTA v2在相当或更低通信预算下持续领先;相比FedMD式全局教师有显著提升。在使用ResNet-18的CIFAR-10上,仅需约FedAvg 1/1100的通信量即可达57.7%测试准确率;在AG News上仅需约1/300的流量即达89.3%准确率。

原文摘要 · Abstract (English)

Federated learning (FL) enables collaborative training over distributed multimedia data but suffers acutely from statistical heterogeneity and communication constraints, especially when clients deploy large models. Classic parameter-averaging methods such as FedAvg transmit full model weights and can diverge under nonindependent and identically distributed (non-IID) data. We propose KTA v2, a prediction-space knowledge trading market for FL. Each round, clients locally train on their private data, then share only logits on a small public reference set. The server constructs a client-client similarity graph in prediction space, combines it with reference-set accuracy to form per-client teacher ensembles, and sends back personalized soft targets for a second-stage distillation update. This two-stage procedure can be interpreted as approximate block-coordinate descent on a unified objective with prediction-space regularization. Experiments on FEMNIST, CIFAR-10 and AG News show that, under comparable or much lower communication budgets, KTA v2 consistently outperforms a local-only baseline and strong parameter-based methods (FedAvg, FedProx), and substantially improves over a FedMD-style global teacher. On CIFAR-10 with ResNet-18, KTA v2 reaches 57.7% test accuracy using approximately 1/1100 of FedAvg's communication, while on AG News it attains 89.3% accuracy with approximately 1/300 of FedAvg's traffic.

联邦学习知识蒸馏通信效率多媒体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。