用特征不兼容性动态决定是否跳过残差块，显著减少计算量。
CosineGate: Semantic Dynamic Routing via Cosine Incompatibility in Residual Networks
- 基于特征余弦不相容性设计可微路由机制，自动判断是否跳过残差块。
- 在CIFAR-10上实现最高93.2%准确率，同时节省最多28.5%计算量。
- 无需额外监督或调参，适用于对效率敏感的图像分类任务。
现代深度残差网络对所有输入都计算全部残差块，造成大量冗余。本文提出CosineGate，一种端到端可微的动态路由架构，利用身份映射与残差特征表示间的余弦不相容性作为自监督跳过信号。通过定义余弦不相容性比率（CIR = 1 - cos(x, F(x))），并采用Gumbel-Softmax松弛实现训练时每样本、每块的门控。引入渐进式FLOPs正则项控制平均计算开销，避免优化不稳定。在CIFAR-10上，激进配置达89.9%准确率且节省24.1% FLOPs，平衡配置在第160轮达91.3%准确率并节省28.5%计算量，保守配置峰值准确率达93.2%，计算量减少最小。结果优于或持平于ResNet-20，无需辅助监督、知识蒸馏或任务特定启发式方法。表明简单的特征几何不相容性可作为动态残差路由的有效信号。
原文摘要 · Abstract (English)
Modern deep residual networks perform substantial redundant computation by evaluating all residual blocks for every input, even when identity mappings suffice. We introduce CosineGate, an end-to-end differentiable architecture for dynamic routing in residual networks that uses cosine incompatibility between identity and residual feature representations as a self-supervised skip signal. CosineGate measures semantic redundancy through the Cosine Incompatibility Ratio (CIR), defined as 1 - cos(x, F(x)), and uses Gumbel-Softmax relaxation to enable per-sample, per-block gating during training. A progressive FLOPs regularization term controls average compute usage without destabilizing optimization. On CIFAR-10, CosineGate spans the accuracy-efficiency Pareto frontier: an aggressive configuration achieves 89.9 percent accuracy with 24.1 percent FLOPs savings, a balanced configuration achieves 91.3 percent accuracy with 28.5 percent savings at epoch 160, and a conservative configuration reaches a peak of 93.2 percent accuracy with minimal compute reduction. These results match or exceed ResNet-20 (91.3 percent) while reducing computation, without auxiliary supervision, distillation, or task-specific heuristics. Our results demonstrate that simple geometric measures of feature incompatibility provide a principled and effective signal for dynamic residual routing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。