提出可操控数据拓扑的新型激活函数，提升低维场景下的网络性能。
Topology-Aware Activation Functions in Neural Networks
- 设计可实现数据拓扑'切割'的SmoothSplit与ParametricSplit函数
- 在低维设置下优于传统激活函数，高维也保持竞争力
- 适合关注数据流形结构与低维表示的研究者
本研究探索了新型激活函数，以增强神经网络在训练过程中操纵数据拓扑的能力。针对传统激活函数（如ReLU）的局限性，提出SmoothSplit和ParametricSplit，引入拓扑'切割'功能。这些函数使网络能更有效地转换复杂数据流形，尤其在低维层中表现优异。在合成数据和真实世界数据集上的实验表明，ParametricSplit在低维设置下表现超越传统激活函数，同时在高维设置中仍保持竞争力。结果表明，拓扑感知激活函数对推进神经网络架构具有潜力。代码已公开于https://github.com/Snopoff/Topology-Aware-Activations。
原文摘要 · Abstract (English)
This study explores novel activation functions that enhance the ability of neural networks to manipulate data topology during training. Building on the limitations of traditional activation functions like $\mathrm{ReLU}$, we propose $\mathrm{SmoothSplit}$ and $\mathrm{ParametricSplit}$, which introduce topology "cutting" capabilities. These functions enable networks to transform complex data manifolds effectively, improving performance in scenarios with low-dimensional layers. Through experiments on synthetic and real-world datasets, we demonstrate that $\mathrm{ParametricSplit}$ outperforms traditional activations in low-dimensional settings while maintaining competitive performance in higher-dimensional ones. Our findings highlight the potential of topology-aware activation functions in advancing neural network architectures. The code is available via https://github.com/Snopoff/Topology-Aware-Activations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。