解决表格数据回归不平衡问题,测试时自适应调整专家权重。
Mixture Experts with Test-Time Self-Supervised Aggregation for Tabular Imbalanced Regression
- 用高斯混合模型划分区域,训练区域专属专家
- 测试时根据输入动态调整专家权重,提升泛化能力
- 在房价、骑行预测等数据集上平均降低7.1%误差
表格数据在金融、城市规划等实际应用中广泛存在。现有研究多关注分类任务中的数据不平衡,而表格回归任务的不平衡问题尚未充分探索,尤其因连续标签边界模糊及对测试分布均衡的假设不成立,导致模型性能下降。为此,我们提出MATI:基于测试时自监督聚合的混合专家方法,包含两项创新:(i) 区域感知混合专家,利用高斯混合模型捕捉潜在区域,通过各分量统计信息构建区域专属专家以学习局部特征;(ii) 测试时自监督专家聚合,根据测试样本特征动态调整专家权重,增强对不同测试分布的适应性。我们在四个真实世界表格不平衡回归数据集(房屋定价、共享单车、年龄预测等)上评估,采用三种测试分布:均匀分布、符合训练数据的正态分布、强调罕见目标区域的逆分布。在三种分布下平均,MATI相比现有方法将MAE降低7.1%。
原文摘要 · Abstract (English)
Tabular data serve as a fundamental and ubiquitous representation of structured information in numerous real-world applications, e.g., finance and urban planning. In the realm of tabular imbalanced applications, data imbalance has been investigated in classification tasks with insufficient instances in certain labels, causing the model's ineffective generalizability. However, the imbalance issue of tabular regression tasks is underexplored, and yet is critical due to unclear boundaries for continuous labels and simplifying assumptions in existing imbalance regression work, which often rely on known and balanced test distributions. Such assumptions may not hold in practice and can lead to performance degradation. To address these issues, we propose MATI: Mixture Experts with Test-Time Self-Supervised Aggregation for Tabular Imbalance Regression, featuring two key innovations: (i) the Region-Aware Mixture Expert, which adopts a Gaussian Mixture Model to capture the underlying related regions. The statistical information of each Gaussian component is then used to synthesize and train region-specific experts to capture the unique characteristics of their respective regions. (ii) Test-Time Self-Supervised Expert Aggregation, which dynamically adjusts region expert weights based on test data features to reinforce expert adaptation across varying test distributions. We evaluated MATI on four real-world tabular imbalance regression datasets, including house pricing, bike sharing, and age prediction. To reflect realistic deployment scenarios, we adopted three types of test distributions: a balanced distribution with uniform target frequencies, a normal distribution that follows the training data, and an inverse distribution that emphasizes rare target regions. On average across these three test distributions, MATI achieved a 7.1% improvement in MAE compared to existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。