arXiv:2501.05809cs.LG2025-01被引 1

通过考虑数据间相对顺序,提升回归模型的准确性和鲁棒性。

AdaPRL: Adaptive Pairwise Regression Learning with Uncertainty Estimation for Universal Regression Tasks

  • 基于成对样本的相对关系进行学习,捕捉数据内在关联
  • 融合不确定性估计,有效降低噪声数据影响,减少过拟合
  • 适用于多任务与时间序列预测,兼容主流网络结构

现有深度回归模型通常以点对点方式学习,忽视不同样本间的相对顺序,导致忽略数据间关系,可能影响性能。同时,训练数据中的随机不确定性会诱导模型学习非泛化模式,加剧过拟合。为此,我们提出一种自适应成对回归学习框架AdaPRL,利用数据点间的相对差异,并结合深度概率模型量化预测不确定性。该方法进一步拓展至多任务学习与多变量时间序列预测。在推荐系统、年龄预测、时间序列、自然语言理解、金融及工业数据等多个真实世界回归数据集上实验表明,AdaPRL兼容多种主干网络,在绝大多数任务中达到当前最优性能,且无需额外推理开销,显著提升预测精度、排序能力、泛化性、抗噪性、小样本适应性与可解释性。实验还显示,它可无缝集成至近期提出的回归框架中实现性能提升。

原文摘要 · Abstract (English)

Current deep regression models usually learn in a point-wise way that treats each sample as an independent input, neglecting the relative ordering among different data. Consequently, the regression model could neglect the data's interrelationships, potentially resulting in suboptimal performance. Moreover, the existence of aleatoric uncertainty in the training data may drive the model to capture non-generalizable patterns, contributing to increased overfitting. To address these issues, we propose a novel adaptive pairwise learning framework for regression tasks (AdaPRL) which leverages the relative differences between data points and integrates with deep probabilistic models to quantify the uncertainty associated with the predictions. Additionally, we adapt AdaPRL for applications in multi-task learning and multivariate time series forecasting. Extensive experiments with several real-world regression datasets including recommendation systems, age prediction, time series forecasting, natural language understanding, finance, and industry datasets show that AdaPRL is compatible with different backbone networks in various tasks and achieves state-of-the-art performance on the vast majority of tasks without extra inference cost, highlighting its notable potential including enhancing prediction accuracy and ranking ability, increasing generalization capability, improving robustness to noisy data, improving resilience to reduced data, and enhancing interpretability. Experiments also show that AdaPRL can be seamlessly incorporated into recently proposed regression frameworks to gain performance improvement.

回归学习不确定性估计成对学习多任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。