arXiv:2605.09256cs.LGcs.AI2026-05

通过随机重连模型副本间的通信路径,提升泛化能力。

Improving Generalization by Permutation Routing Across Model Copies

论文配图:Improving Generalization by Permutation Routing Across Model Copies
图 1 · 摘自论文原文
  • 用置换路由替代参数平均,让各模型副本间信息交换更灵活
  • 在感知机、委员会机和多层网络上均验证了泛化性能提升
  • 适合关注模型泛化与分布式训练机制的研究者

我们引入了M-覆盖(或M-层)变换用于机器学习。该方法将模型复制M次,但不通过参数平均或显式吸引力耦合副本,而是重新配置局部学习消息的计算上下文。每个局部损失在由来自不同副本的参数按置换采样构成的路由模型上评估,训练仍使用原始局部更新规则,而学习消息则通过这些路由计算路径在副本间重新分配。因此,结构化混合核Q定义了消息传输拓扑,并控制升维因子图的长环结构。我们在感知机、委员会机和多层感知机上构建此框架,表明该原理适用于离散模型到可微神经网络。该框架通过结构化消息共享机制,而非副本坍缩或参数空间耦合,实现了泛化能力的提升。

原文摘要 · Abstract (English)

We introduce a use of the \(M\)-cover (or \(M\)-layer) transform for machine learning. The method replicates a model \(M\) times, but instead of coupling the copies through parameter averaging or an explicit attractive force, as in replicated SGD or Elastic SGD, it rewires the contexts in which local learning messages are computed. Each local loss is evaluated on a routed model whose parameters are drawn from different copies according to permutations sampled from a structured mixing kernel \(Q\). Training then uses the original local update rule, while the resulting learning messages are redistributed across the copies through these routed computational paths. Thus \(Q\) defines a topology for message transport and controls the long-loop structure of the lifted factor graph. We formulate this construction for perceptrons, committee machines, and multilayer perceptrons, showing that the same principle applies from discrete models to differentiable neural networks. The resulting framework provides a mechanism for improving generalization through structured message sharing rather than replica collapse or parameter-space coupling.

模型泛化分布式训练消息传递深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。