提出新型双曲残差网络LResNet,提升层次数据建模的稳定性与效率。
Lorentzian Residual Neural Networks
- 基于洛伦兹模型的加权中心点构建残差连接
- 在图与视觉任务中超越主流欧氏与双曲方法
- 适用于CNN、GNN、图Transformer等多种架构
双曲神经网络已成为建模现实数据中普遍存在的层次结构的有效工具。残差连接通过层间直接信息传递,对深度神经网络的成功至关重要。然而,现有双曲残差网络存在模型复杂度高、数值不稳定性及多次映射到切空间带来的误差等问题。为此,我们提出LResNet,一种基于洛伦兹模型中加权洛伦兹中心点的新型洛伦兹残差神经网络。该方法实现了洛伦兹双曲神经网络中残差连接的高效集成,同时保持其层次化表征能力。理论上可推导出先前方法,并展现出更高的稳定性、效率与有效性。在图与视觉任务上的大量实验表明,本方法显著优于当前最优的欧氏与双曲基线。研究结果凸显了LResNet在双曲嵌入空间中构建更富表达力神经网络的潜力,是一种可广泛应用于CNN、GNN和图Transformer等架构的通用方法。
原文摘要 · Abstract (English)
Hyperbolic neural networks have emerged as a powerful tool for modeling hierarchical data structures prevalent in real-world datasets. Notably, residual connections, which facilitate the direct flow of information across layers, have been instrumental in the success of deep neural networks. However, current methods for constructing hyperbolic residual networks suffer from limitations such as increased model complexity, numerical instability, and errors due to multiple mappings to and from the tangent space. To address these limitations, we introduce LResNet, a novel Lorentzian residual neural network based on the weighted Lorentzian centroid in the Lorentz model of hyperbolic geometry. Our method enables the efficient integration of residual connections in Lorentz hyperbolic neural networks while preserving their hierarchical representation capabilities. We demonstrate that our method can theoretically derive previous methods while offering improved stability, efficiency, and effectiveness. Extensive experiments on both graph and vision tasks showcase the superior performance and robustness of our method compared to state-of-the-art Euclidean and hyperbolic alternatives. Our findings highlight the potential of LResNet for building more expressive neural networks in hyperbolic embedding space as a generally applicable method to multiple architectures, including CNNs, GNNs, and graph Transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。