揭示长尾半监督学习中逻辑值偏倚的动态机制，提出新方法提升模型泛化能力。
Learning Dynamics of Logits Debiasing for Long-Tailed Semi-Supervised Learning
- 通过梯度动态分解发现逻辑值偏倚源于类别不平衡
- 提出DyTrim框架，动态调整有标签与无标签数据的梯度分配
- 理论保证减少类别偏倚，适用于各类长尾半监督场景
现实中的半监督学习常面临长尾分布问题，伪标签易偏向多数类，导致泛化性能下降。尽管已有多种长尾半监督学习（LTSSL）方法，但其隐式校正逻辑值的机制仍不清晰。本文从学习动态角度重新审视LTSSL，推导出逻辑值更新的分步分解，表明预测主要受类别不平衡偏倚主导，且该偏倚稳定反映标签先验。为揭示此效应，我们使用任务无关基线图像的逻辑值作为累积偏倚指标，证明其收敛于类别先验。这一发现统一了逻辑值调整、重加权和重采样等方法的本质——重塑梯度动态。基于此，我们提出基于原则的动态剪枝框架DyTrim：对有标签数据进行类别感知剪枝，对无标签数据采用置信度软剪枝，重新分配梯度预算。理论证明DyTrim可降低类别偏倚并提升泛化性能。在标准LTSSL基准上的大量实验表明，DyTrim在不同架构和方法上均取得一致提升。代码已公开：https://jiajun0425.github.io/DyTrim。
原文摘要 · Abstract (English)
Long-tailed distributions are prevalent in real-world semi-supervised learning (SSL), where pseudo-labels tend to favor majority classes, leading to degraded generalization. While many long-tailed semi-supervised learning (LTSSL) methods have been proposed, the mechanisms by which they implicitly debias logits remain poorly understood. In this work, we revisit LTSSL through the lens of learning dynamics and provide a theoretical characterization of logits debiasing. Specifically, we derive a step-wise decomposition of the logits updates, showing that predictions are dominated by class-imbalance bias that reliably reflects label priors. To expose this effect, we use the logits of a task-irrelevant baseline image as an indicator of accumulated bias and prove that they converge to the class prior. This provides a unified view where LTSSL remedies such as logit adjustment, reweighting, and resampling correspond to reshaping gradient dynamics. Based on this insight, we propose DyTrim, a principle-based dynamic pruning framework that reallocates gradient budget through class-aware pruning on labeled data and confidence-based soft pruning on unlabeled data. We provide theoretical guarantees that DyTrim reduces class bias and improves generalization. Extensive experiments on standard LTSSL benchmarks show consistent gains across architectures and methods. Code available at: https://jiajun0425.github.io/DyTrim
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。