用可解释的量化估计模型实现概率回归的迁移学习，提升预测性能。
Probabilistic Pretraining for Neural Regression
- 基于排列不变性设计可解释的量化估计模型
- 在多个下游数据集预训练后微调，显著提升单任务表现
- 在Kaggle竞赛中优于树模型和最新神经基座模型
概率回归的迁移学习仍缺乏探索。本文提出NIAQUE（Neural Interpretable Any-Quantile Estimation），一种通过排列不变性设计的新模型，用于概率回归中的迁移学习。我们证明，在多样化的下游回归数据集上直接预训练NIAQUE，并在特定目标数据集上微调，能有效提升单个回归任务的性能，展示了概率迁移学习的积极影响。此外，与包含树模型及近期神经基座模型TabPFN、TabDPT的强基线相比，NIAQUE在Kaggle竞赛中表现出色。结果表明，NIAQUE是一种稳健且可扩展的概率回归框架，利用迁移学习显著增强预测能力。
原文摘要 · Abstract (English)
Transfer learning for probabilistic regression remains underexplored. This work closes this gap by introducing NIAQUE, Neural Interpretable Any-Quantile Estimation, a new model designed for transfer learning in probabilistic regression through permutation invariance. We demonstrate that pre-training NIAQUE directly on diverse downstream regression datasets and fine-tuning it on a specific target dataset enhances performance on individual regression tasks, showcasing the positive impact of probabilistic transfer learning. Furthermore, we highlight the effectiveness of NIAQUE in Kaggle competitions against strong baselines involving tree-based models and recent neural foundation models TabPFN and TabDPT. The findings highlight NIAQUE's efficacy as a robust and scalable framework for probabilistic regression, leveraging transfer learning to enhance predictive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。