提出可跨模型通用的数据子集选择框架，提升训练效率与泛化能力。
Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networks
- 用基于图结构的注意力机制构建模型预测代理，替代实际训练。
- 两种变体：快速优化选子集（横贯式）和直接推理选子集（归纳式）。
- 在多个数据集上优于现有方法，适合不同架构的高效训练场景。
现有高效学习中的子集选择方法多采用离散组合与模型特定策略，缺乏跨架构泛化能力——对未见过的网络结构无法复用已选子集。为此，本文提出可训练的子集选择框架 $ exttt{SubSelNet}$，实现跨架构泛化。首先引入一种基于注意力的神经组件，利用网络架构的图结构作为代理模型，快速预测性能。随后基于这些预测构建子集采样器，自然衍生出两种变体：横贯式 $ exttt{SubSelNet}$ 为每个模型求解小型优化问题，速度仍极快；归纳式 $ exttt{SubSelNet}$ 则通过预训练的选择器直接生成子集，无需优化。实验表明，该方法在多个真实数据集上均优于多种基线方法。
原文摘要 · Abstract (English)
Existing subset selection methods for efficient learning predominantly employ discrete combinatorial and model-specific approaches which lack generalizability. For an unseen architecture, one cannot use the subset chosen for a different model. To tackle this problem, we propose $\texttt{SubSelNet}$, a trainable subset selection framework, that generalizes across architectures. Here, we first introduce an attention-based neural gadget that leverages the graph structure of architectures and acts as a surrogate to trained deep neural networks for quick model prediction. Then, we use these predictions to build subset samplers. This naturally provides us two variants of $\texttt{SubSelNet}$. The first variant is transductive (called as Transductive-$\texttt{SubSelNet}$) which computes the subset separately for each model by solving a small optimization problem. Such an optimization is still super fast, thanks to the replacement of explicit model training by the model approximator. The second variant is inductive (called as Inductive-$\texttt{SubSelNet}$) which computes the subset using a trained subset selector, without any optimization. Our experiments show that our model outperforms several methods across several real datasets
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。