联合训练让模型在部署时更准，避免额外信息误导。
Coupled Training with Privileged Information and Unlabeled Data
- 两阶段训练改为联合优化，让部署模型只在有用时参考额外信息。
- 实验证明该方法在合成与真实数据上均显著优于传统两阶段方法。
- 适合有昂贵训练数据但部署时不可用的场景，如医疗或工业预测。
在许多预测任务中，训练阶段可获得额外信息（如成本高或耗时的测量），但部署时无法获取。传统做法是先用全部信息训练一个模型，再用其对无标签样本的预测结果训练第二个仅依赖测试输入的模型。然而，当额外信息弱或含噪声时，这种两阶段方法可能误导部署模型，甚至降低准确率。本文提出一种联合训练方法，同时学习两个模型，使部署模型仅在额外信息确实有帮助时才利用它，避免继承错误。我们提供了理论保证，说明联合训练何时能提升预测性能，并分析了适用于大规模高维模型的简单交替训练算法。在合成数据和真实预测任务上的实验表明，该方法能有效避免失败，稳健优于标准两阶段基线。
原文摘要 · Abstract (English)
In many prediction problems, we have extra information during training (for example, measurements that are expensive or slow to collect) that will not be available when the model is deployed. A common strategy is to first train a model that uses all training information, then use its predictions on unlabeled examples to train a second model that only uses the inputs available at test time. However, when the extra training-only information is weak or noisy, this Two-Stage approach can mislead the deployment model and even hurt accuracy. We propose a joint training method that learns the two models together, so the deployment model can benefit from the extra information only when it actually helps, instead of inheriting its mistakes. We provide guarantees that describe when joint training improves prediction accuracy and analyze a simple alternating training algorithm for large, high-dimensional models. Experiments on synthetic data and real-world prediction tasks show that our approach avoids these failures and robustly outperforms standard Two-Stage baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。