arXiv:2510.01706cs.LGcs.AI2025-10被引 3

用全局优化方法对齐模型层与脑区表征,解决深度不同时的匹配难题。

Representational Alignment Across Model Layers and Brain Regions with Multi-Level Optimal Transport

  • 通过多层级最优传输实现层间软映射与神经元级运输规划
  • 在视觉模型、大语言模型和人脑皮层数据中均优于传统方法
  • 可发现层次化结构对应关系,适合跨架构模型对比研究

标准表征相似性方法独立地将网络每一层与另一网络的最佳匹配层对齐,导致结果不对称、缺乏全局对齐评分,并难以处理深度不同的网络。这些问题源于忽略全局激活结构,且限制映射为刚性的单对一对应。本文提出多层级最优传输(MOT)框架,联合推断软的、全局一致的层间耦合与神经元级运输方案。源神经元可将信息分布至多个目标层,同时在边际约束下最小化总运输成本。该方法既提供全网络比较的单一对齐分数,又通过质量分配自然解决深度不匹配问题。我们在视觉模型、大语言模型及人类视觉皮层记录数据上评估MOT,结果表明其在所有领域中对齐质量达到或超过标准成对匹配。此外,它揭示了平滑、精细的分层对应关系:早期层对应早期层,深层保持相对位置,深度差异通过跨多层分布表示来解决。这些结构化模式由全局优化自然产生,而非人为施加,而在贪心逐层方法中则不存在。MOT因此实现了更丰富、更具解释性的表征比较,尤其适用于架构或深度不同的网络。我们进一步扩展方法为三层级MOT框架,验证了两网络训练轨迹间的对齐可行性,证明了MOT能发现贪心逐层匹配所遗漏的检查点级对应关系。

原文摘要 · Abstract (English)

Standard representational similarity methods align each layer of a network to its best match in another independently, producing asymmetric results, lacking a global alignment score, and struggling with networks of different depths. These limitations arise from ignoring global activation structure and restricting mappings to rigid one-to-one layer correspondences. We propose Multi-Level Optimal Transport (MOT), a unified framework that jointly infers soft, globally consistent layer-to-layer couplings and neuron-level transport plans. MOT allows source neurons to distribute mass across multiple target layers while minimizing total transport cost under marginal constraints. This yields both a single alignment score for the entire network comparison and a soft transport plan that naturally handles depth mismatches through mass distribution. We evaluate MOT on vision models, large language models, and human visual cortex recordings. Across all domains, MOT matches or surpasses standard pairwise matching in alignment quality. Moreover, it reveals smooth, fine-grained hierarchical correspondences: early layers map to early layers, deeper layers maintain relative positions, and depth mismatches are resolved by distributing representations across multiple layers. These structured patterns emerge naturally from global optimization without being imposed, yet are absent in greedy layer-wise methods. MOT thus enables richer, more interpretable comparisons between representations, particularly when networks differ in architecture or depth. We further extend our method to a three-level MOT framework, providing a proof-of-concept alignment of two networks across their training trajectories and demonstrating that MOT uncovers checkpoint-wise correspondences missed by greedy layer-wise matching.

表征对齐最优传输深度学习脑科学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。