用迁移学习+集成模型,智能选初始数据,省标签成本
Active Transfer Bagging: A New Approach for Accelerated Active Learning Acquisition of Data by Combined Transfer Learning and Bagging Based Models
- 基于袋装模型的贝叶斯分析,量化候选数据的信息量
- 在10-100个初始样本下,学习曲线提升显著,低数据时优势最大
- 适合标注昂贵、有相似数据集的场景,如气象、金融、化学
现代机器学习虽成果丰硕,但常依赖大量标注数据。主动学习可降低标注成本,但早期性能受限于随机初始化的数据集。若存在相关或近似数据集,可用来构建更优初始集。本文提出主动迁移袋装法(ATBagging),通过比较袋内与袋外预测分布,从贝叶斯视角估计候选数据的信息量,作为信息增益代理。为避免重复选择,引入基于随机傅里叶特征和质量-多样性因子分解的确定性点过程(DPP)进行采样。该方法同时用于主动学习阶段新增样本的选择。在四组真实数据集(QM9、ERA5、Forbes 2000、北京PM2.5)上评估,当初始样本数为10–100时,ATBagging在几乎所有情况下均优于或持平现有方法,尤其在低数据场景下提升明显。表明ATBagging是一种低成本、高回报的主动学习启动方案。
原文摘要 · Abstract (English)
Modern machine learning has achieved remarkable success on many problems, but this success often depends on the existence of large, labeled datasets. While active learning can dramatically reduce labeling cost when annotations are expensive, early performance is frequently dominated by the initial seed set, typically chosen at random. In many applications, however, related or approximate datasets are readily available and can be leveraged to construct a better seed set. We introduce a new method for selecting the seed data set for active learning, Active-Transfer Bagging (ATBagging). ATBagging estimates the informativeness of candidate data point from a Bayesian interpretation of bagged ensemble models by comparing in-bag and out-of-bag predictive distributions from the labeled dataset, yielding an information-gain proxy. To avoid redundant selections, we impose feature-space diversity by sampling a determinantal point process (DPP) whose kernel uses Random Fourier Features and a quality-diversity factorization that incorporates the informativeness scores. This same blended method is used for selection of new data points to collect during the active learning phase. We evaluate ATBagging on four real-world datasets covering both target-transfer and feature-shift scenarios (QM9, ERA5, Forbes 2000, and Beijing PM2.5). Across seed sizes nseed = 10-100, ATBagging improves or ties early active learning and increases area under the learning-curve relative to alternative seed subset selection methodologies in almost all cases, with strongest benefits in low-data regimes. Thus, ATBagging provides a low-cost, high reward means to initiating active learning-based data collection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。