用博弈论框架实现异构数据下的生成模型分布式学习
Game-theoretic distributed learning of generative models for heterogeneous data collections
- 以生成合成数据替代参数共享，处理异构模型与数据
- 证明指数族模型存在唯一纳什均衡且算法可收敛
- 适用于多模态数据，适合隐私敏感的分布式场景
分布式学习面临异构本地模型和数据的挑战。基于生成模型的成功，我们提出通过交换合成数据而非共享模型参数来应对该问题。本地模型被视为黑箱，能从数据中学习参数并生成数据。若本地模型支持半监督学习，可扩展至不同概率空间，从而处理多模态异构数据。我们将本地模型学习建模为合作博弈，基于博弈论原理。证明指数族本地模型存在唯一纳什均衡，并显示所提方法收敛于该均衡。在标准视觉基准数据集上验证了该方法在图像分类和条件生成任务中的优势。
原文摘要 · Abstract (English)
One of the main challenges in distributed learning arises from the difficulty of handling heterogeneous local models and data. In light of the recent success of generative models, we propose to meet this challenge by building on the idea of exchanging synthetic data instead of sharing model parameters. Local models can then be treated as ``black boxes'' with the ability to learn their parameters from data and to generate data according to these parameters. Moreover, if the local models admit semi-supervised learning, we can extend the approach by enabling local models on different probability spaces. This allows to handle heterogeneous data with different modalities. We formulate the learning of the local models as a cooperative game starting from the principles of game theory. We prove the existence of a unique Nash equilibrium for exponential family local models and show that the proposed learning approach converges to this equilibrium. We demonstrate the advantages of our approach on standard benchmark vision datasets for image classification and conditional generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。