为不同数据设计专用子网络，提升模型效率与准确性
Routing the Lottery: Adaptive Subnetworks for Heterogeneous Data
- 根据数据类别或环境动态选择最优子网络
- 在多种任务上准确率和召回率均优于基线，参数量少10倍
- 可无标签诊断过度剪枝问题，适合资源受限场景
剪枝中的彩票理论假设大模型中存在稀疏子网络（即‘中奖票’），可独立训练并达到与完整模型相当的性能。然而现有方法多假设所有输入共享同一张‘中奖票’，忽视真实数据的异质性。本文提出路由彩票（RTL）框架，发现多个针对特定类别、语义簇或环境条件的自适应子网络（自适应票）。在多个数据集和任务上，RTL在平衡准确率与召回率上持续优于单/多模型基线，参数量仅为独立模型的1/10，且语义对齐。此外，我们识别出子网络坍缩现象（极端剪枝导致性能下降），并引入子网络相似度评分，实现无标签诊断过度剪枝。结果表明，剪枝可作为模型结构与数据异质性对齐的机制，推动更模块化、上下文感知的深度学习发展。
原文摘要 · Abstract (English)
In pruning, the Lottery Ticket Hypothesis posits that large networks contain sparse subnetworks, or winning tickets, that can be trained in isolation to match the performance of their dense counterparts. However, most existing approaches assume a single universal winning ticket shared across all inputs, ignoring the inherent heterogeneity of real-world data. In this work, we propose Routing the Lottery (RTL), an adaptive pruning framework that discovers multiple specialized subnetworks, called adaptive tickets, each tailored to a class, semantic cluster, or environmental condition. Across diverse datasets and tasks, RTL consistently outperforms single- and multi-model baselines in balanced accuracy and recall, while using up to 10 times fewer parameters than independent models and exhibiting semantically aligned. Furthermore, we identify subnetwork collapse, a performance drop under aggressive pruning, and introduce a subnetwork similarity score that enables label-free diagnosis of oversparsification. Overall, our results recast pruning as a mechanism for aligning model structure with data heterogeneity, paving the way toward more modular and context-aware deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。