用双曲空间提升早期退出网络的准确率和可靠性
More Than A Shortcut: A Hyperbolic Approach To Early-Exit Networks
- 在双曲空间中构建层级表示,确保深层特征精确细化浅层
- 早期退出阶段性能提升显著,计算成本更低
- 可量化不确定性,适合资源受限设备部署
在资源受限设备上部署高精度事件检测面临性能与计算成本之间的权衡。早期退出(EE)网络通过自适应计算提供解决方案,但通常无法保证一致的层级结构,限制了早期预测的可靠性。为此,我们提出双曲早期退出网络(HypEE),一种在双曲空间中学习EE表示的新框架。核心贡献是包含新型蕴含损失的层级训练目标,强制施加偏序约束,确保深层网络层在几何上逐步优化浅层表示。在多个音频事件检测任务和主干架构上的实验表明,HypEE显著优于标准欧氏空间的EE基线,尤其在最早、最耗时的退出阶段表现突出。学习到的几何结构还提供了可靠的不确定性度量,支持一种新颖的触发机制,使系统整体比传统EE和无早期退出的标准主干模型更高效、更准确。
原文摘要 · Abstract (English)
Deploying accurate event detection on resource-constrained devices is challenged by the trade-off between performance and computational cost. While Early-Exit (EE) networks offer a solution through adaptive computation, they often fail to enforce a coherent hierarchical structure, limiting the reliability of their early predictions. To address this, we propose Hyperbolic Early-Exit networks (HypEE), a novel framework that learns EE representations in the hyperbolic space. Our core contribution is a hierarchical training objective with a novel entailment loss, which enforces a partial-ordering constraint to ensure that deeper network layers geometrically refine the representations of shallower ones. Experiments on multiple audio event detection tasks and backbone architectures show that HypEE significantly outperforms standard Euclidean EE baselines, especially at the earliest, most computationally-critical exits. The learned geometry also provides a principled measure of uncertainty, enabling a novel triggering mechanism that makes the overall system both more efficient and more accurate than a conventional EE and standard backbone models without early-exits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。