用多重宇宙思想设计分支神经网络，单头表现优于基准模型。
ANDHRA Bandersnatch: Training Neural Networks to Predict Parallel Realities
- 输入信号在每层分裂为多条独立路径，形成并行预测头。
- 三层分支因子为2时生成8个输出头，某头准确率显著超越基线。
- 推理时无需额外计算，适合追求高精度的轻量级应用。
受多世界诠释启发，本文提出一种新型神经网络架构，将相同输入信号在每一层分裂为多个分支，采用超修正激活函数（ANDHRA）。分支层不合并，形成独立网络路径，最终产生多个输出头进行预测。以三层分支因子为2为例，总共有2^3 = 8个输出头。各头通过合并损失值联合训练。该架构因增加分支需更多参数和内存。但在CIFAR-10/100上的实验表明，推理阶段存在单一头的表现优于基线，在参数与计算成本相同的情况下实现统计显著提升。
原文摘要 · Abstract (English)
Inspired by the Many-Worlds Interpretation (MWI), this work introduces a novel neural network architecture that splits the same input signal into parallel branches at each layer, utilizing a Hyper Rectified Activation, referred to as ANDHRA. The branched layers do not merge and form separate network paths, leading to multiple network heads for output prediction. For a network with a branching factor of 2 at three levels, the total number of heads is 2^3 = 8 . The individual heads are jointly trained by combining their respective loss values. However, the proposed architecture requires additional parameters and memory during training due to the additional branches. During inference, the experimental results on CIFAR-10/100 demonstrate that there exists one individual head that outperforms the baseline accuracy, achieving statistically significant improvement with equal parameters and computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。