arXiv:2603.21056cs.LG2026-03

通过平衡类别角度方差,提升少样本下的文本分类伪标签准确率。

Semi-Supervised Learning with Balanced Deep Representation Distributions

  • 用角度余弦损失和高斯变换平衡各类别内部表示角度方差
  • 在自训练中显著提升未标注数据伪标签准确率,尤其在标签稀缺时
  • 适用于小样本场景的多分类与多标签文本分类任务

半监督文本分类(SSTC)通常采用自训练范式:先用有标签文本训练深度分类器,再交替预测无标签文本的伪标签,并用标签与伪标签混合数据继续训练。然而,其性能受伪标签准确性影响显著,而现有方法常因标签间表示分布差异过大导致边际偏差问题。为此,本文提出S2TC-BDD方法,通过引入角度余弦损失并施加若干高斯线性变换,实现各类别内部文本表示角度方差的均衡化,即在自训练循环中对有标签与伪标签文本联合估计并约束标签角度方差平衡。该机制有效提升了伪标签质量。我们实现了S2TC-BDD的多分类与多标签版本,通过伪标签策略与正则项优化。实验表明,该方法在标签稀疏条件下优于当前主流方法,验证了其有效性。

原文摘要 · Abstract (English)

Semi-Supervised Text Classification (SSTC) mainly works under the spirit of self-training. They initialize the deep classifier by training over labeled texts; and then alternatively predict unlabeled texts as their pseudo-labels and train the deep classifier over the mixture of labeled and pseudo-labeled texts. Naturally, their performance is largely affected by the accuracy of pseudo-labels for unlabeled texts. Unfortunately, they often suffer from low accuracy because of the margin bias problem caused by the large difference between representation distributions of labels in SSTC. To alleviate this problem, we apply the angular margin loss, and perform several Gaussian linear transformations to achieve balanced label angle variances, i.e., the variance of label angles of texts within the same label. More accuracy of predicted pseudo-labels can be achieved by constraining all label angle variances balanced, where they are estimated over both labeled and pseudo-labeled texts during self-training loops. With this insight, we propose a novel SSTC method, namely Semi-Supervised Text Classification with Balanced Deep representation Distributions (S2TC-BDD). We implement both multi-class classification and multi-label classification versions of S2TC-BDD by introducing some pseudo-labeling tricks and regularization terms. To evaluate S2 TC-BDD, we compare it against the state-of-the-art SSTC methods. Empirical results demonstrate the effectiveness of S2 TC-BDD, especially when the labeled texts are scarce.

半监督学习文本分类伪标签表示学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。