用谱排序从无标签数据中恢复有序关系,提升半监督回归性能。
Contrastive Learning for Semi-Supervised Deep Regression with Generalized Ordinal Rankings from Spectral Seriation
- 结合有标签与无标签样本构建特征相似矩阵,利用谱排序恢复无标签数据的有序关系。
- 在多个数据集上超越现有最优半监督回归方法,显著降低对标注数据依赖。
- 适合标注成本高、数据量大的回归任务,如生物医学或工业检测场景。
对比学习通过在特征空间中强化标签距离关系来提升回归模型的表示能力,但其高度依赖标签信息以正确恢复特征间的序关系,限制了其在半监督回归中的应用。本文扩展对比回归方法,使无标签数据可在半监督设置下被有效利用,从而减少对昂贵标注的依赖。具体地,我们在一个小型批量中结合有标签与无标签样本构建特征相似矩阵,并通过谱排序算法恢复其中无标签样本的精确序关系,前提是误差水平在一定范围内。有标签样本提供的真值信息为序关系提供正则化,使其更可靠。为进一步减少特征扰动,我们采用动态规划算法选择鲁棒特征用于矩阵构造。恢复的序关系用于对无标签样本进行对比学习,从而扩大可用于特征表示学习的数据规模,实现更鲁棒的结果。此外,该序关系还可作为额外训练信号监督无标签样本的预测。我们在多个数据集上提供了理论保证与实证验证,结果表明本方法优于现有最先进半监督深度回归方法。代码已开源:https://github.com/xmed-lab/CLSS。
原文摘要 · Abstract (English)
Contrastive learning methods enforce label distance relationships in feature space to improve representation capability for regression models. However, these methods highly depend on label information to correctly recover ordinal relationships of features, limiting their applications to semi-supervised regression. In this work, we extend contrastive regression methods to allow unlabeled data to be used in the semi-supervised setting, thereby reducing the dependence on costly annotations. Particularly we construct the feature similarity matrix with both labeled and unlabeled samples in a mini-batch to reflect inter-sample relationships, and an accurate ordinal ranking of involved unlabeled samples can be recovered through spectral seriation algorithms if the level of error is within certain bounds. The introduction of labeled samples above provides regularization of the ordinal ranking with guidance from the ground-truth label information, making the ranking more reliable. To reduce feature perturbations, we further utilize the dynamic programming algorithm to select robust features for the matrix construction. The recovered ordinal relationship is then used for contrastive learning on unlabeled samples, and we thus allow more data to be used for feature representation learning, thereby achieving more robust results. The ordinal rankings can also be used to supervise predictions on unlabeled samples, serving as an additional training signal. We provide theoretical guarantees and empirical verification through experiments on various datasets, demonstrating that our method can surpass existing state-of-the-art semi-supervised deep regression methods. Our code have been released on https://github.com/xmed-lab/CLSS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。