arXiv:2501.06240cs.LGmath.OC2025-01被引 3

证明了胶囊网络动态路由算法的收敛性,解决其效果不稳定问题。

The Convergence of Dynamic Routing between Capsules

  • 将动态路由建模为带线性约束的非线性梯度优化,数学上证明其收敛
  • 发现路由算法在迭代中常仅放大连接强度,不改变分类结果
  • 为提升胶囊网络泛化能力提供理论支撑,适合研究模型稳定性的学者

胶囊网络(CapsNet)是近年来提出的新型神经网络模型,专注于图像中实体的表示与发现。尽管其在泛化能力上优于传统网络,但实验结果却存在矛盾。初步实验表明,路由算法的行为并不总如预期产生良好效果,多数情况下不同路由算法仅使连接强度极化,且持续迭代无变化。为实现CapsNet真正潜力,对路由算法进行深入数学分析至关重要。本文给出了动态路由算法所最小化的目标函数,其为凹函数;该算法可视为在线性约束下求解优化问题的非线性梯度方法,其收敛性可严格数学证明。进一步,我们对这类迭代路由过程的收敛性提供了严谨证明,并详细分析了目标函数与约束间的关系,通过对应路由实验验证了收敛性分析的有效性。

原文摘要 · Abstract (English)

Capsule networks(CapsNet) are recently proposed neural network models with new processing layers, specifically for entity representation and discovery of images. It is well known that CapsNet have some advantages over traditional neural networks, especially in generalization capability. At the same time, some studies report negative experimental results. The causes of this contradiction have not been thoroughly analyzed. The preliminary experimental results show that the behavior of routing algorithms does not always produce good results as expected, and in most cases, different routing algorithms do not change the classification results, but simply polarize the link strength, especially when they continue to repeat without stopping. To realize the true potential of the CapsNet, deep mathematical analysis of the routing algorithms is crucial. In this paper, we will give the objective function that is minimized by the dynamic routing algorithm, which is a concave function. The dynamic routing algorithm can be regarded as nonlinear gradient method to solving an optimization algorithm under linear constraints, and its convergence can be strictly proved mathematically. Furthermore, the mathematically rigorous proof of the convergence is given for this class of iterative routing procedures. We analyze the relation between the objective function and the constraints solved by the dynamic routing algorithm in detail, and perform the corresponding routing experiment to analyze the effect of our convergence proof.

胶囊网络动态路由收敛性优化理论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。