过参数线性模型中，分类任务可经少量回归数据迁移至回归任务。
Task Shift: From Classification to Regression in Overparameterized Linear Models
- 从分类转为回归时，用最小范数插值法分析参数变化机制。
- 零样本下无法迁移；少样本下新后处理算法可渐近恢复真实预测器。
- 揭示了分类器参数的结构化衰减，适合研究迁移学习理论者阅读。
现代机器学习方法在任务迁移中展现出惊人泛化能力，即在相似数据分布下将潜在知识迁移到不同且通常更难的任务。本文研究过参数线性回归设置下的这一现象，其中训练阶段为分类任务，评估阶段为回归任务。在零样本情形（无回归数据）下，我们证明无论信号稀疏或随机，对任意高斯协变量分布，任务迁移均不可能实现。在少样本情形（有少量回归数据）下，我们提出一种简单后处理算法，可渐近恢复真实预测器。分析依赖于最小范数插值引发的参数精细刻画，该结果可能具有独立价值。结果显示，尽管分类任务的最小范数解原本无法直接迁移到回归任务，但其参数呈现意外的结构性衰减，使在有限额外数据下仍可成功完成任务迁移。
原文摘要 · Abstract (English)
Modern machine learning methods have recently demonstrated remarkable capability to generalize under task shift, where latent knowledge is transferred to a different, often more difficult, task under a similar data distribution. We investigate this phenomenon in an overparameterized linear regression setting where the task shifts from classification during training to regression during evaluation. In the zero-shot case, wherein no regression data is available, we prove that task shift is impossible in both sparse signal and random signal models for any Gaussian covariate distribution. In the few-shot case, wherein limited regression data is available, we propose a simple postprocessing algorithm which asymptotically recovers the ground-truth predictor. Our analysis leverages a fine-grained characterization of individual parameters arising from minimum-norm interpolation which may be of independent interest. Our results show that while minimum-norm interpolators for classification cannot transfer to regression a priori, they experience surprisingly structured attenuation which enables successful task shift with limited additional data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。