arXiv:2411.18254cs.LG2024-11

通过模型竞争自动划分数据功能模式,提升回归任务性能

Active partitioning: inverting the paradigm of active learning

  • 多模型竞争预测,优胜者获训练奖励,推动各自专长发展
  • 在20多个回归任务上实现最高54%的损失降低
  • 适合需要精细数据结构解析与模块化建模的应用场景

数据集常包含不同功能模式,分布不均。本文提出一种新型通用划分算法——主动划分(active partitioning),利用多模型迭代预测并竞争,对每个数据点选择最优预测结果进行奖励训练。该机制强化各模型优势,促进其在不同模式上的专业化。专业化的结果可转化为数据划分方案。与传统主动学习聚焦弱点不同,本方法强化模型特长,形成专精分工。我们在含明显功能模式的数据集(如多孔结构的应力-应变数据)上验证该方法,揭示了数据内在结构。作为应用示例,构建由多个专家模型组成的模块化系统,每个模型仅学习一个子分区,并在超过20个流行回归任务中对比单模型全数据训练。结果表明,最大损失降低达54%,证实了该划分算法的有效性。

原文摘要 · Abstract (English)

Datasets often incorporate various functional patterns related to different aspects or regimes, which are typically not equally present throughout the dataset. We propose a novel, general-purpose partitioning algorithm that utilizes competition between models to detect and separate these functional patterns. This competition is induced by multiple models iteratively submitting their predictions for the dataset, with the best prediction for each data point being rewarded with training on that data point. This reward mechanism amplifies each model's strengths and encourages specialization in different patterns. The specializations can then be translated into a partitioning scheme. The amplification of each model's strengths inverts the active learning paradigm: while active learning typically focuses the training of models on their weaknesses to minimize the number of required training data points, our concept reinforces the strengths of each model, thus specializing them. We validate our concept -- called active partitioning -- with various datasets with clearly distinct functional patterns, such as mechanical stress and strain data in a porous structure. The active partitioning algorithm produces valuable insights into the datasets' structure, which can serve various further applications. As a demonstration of one exemplary usage, we set up modular models consisting of multiple expert models, each learning a single partition, and compare their performance on more than twenty popular regression problems with single models learning all partitions simultaneously. Our results show significant improvements, with up to 54% loss reduction, confirming our partitioning algorithm's utility.

主动学习数据划分模块化模型回归任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。