解决长尾数据中模型对头尾类的性能权衡问题
LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups
- 用子采样模型平均减少头部类别偏差
- 在多种长尾分布下实现头尾类性能均衡
- 适合处理不平衡数据集的微调场景
真实世界数据集通常呈现长尾分布，少数头部类别占主导，大量尾部类别严重不足。尽管近期研究表明参数高效微调（PEFT）方法如LoRA和AdaptFormer能保持基础模型（如CLIP）在尾部类别的性能，但代价是头部类别准确率下降。我们发现头尾类比例（η）是影响这一权衡的关键但被忽视的因素。通过在CIFAR100上控制失衡比（ρ）和头尾比（η）的实验，发现PEFT在尾部占优时表现优异，但在更平衡或头部占优的分布中性能下降。为此，提出LT-Soups，一种两阶段模型平均框架：第一阶段对平衡子集微调的模型进行平均以降低头部偏差；第二阶段仅微调分类器在全数据集上恢复头部准确率。在六个基准数据集上的实验表明，LT-Soups在多种失衡环境下均优于PEFT和传统模型平均，取得更优的性能权衡。
原文摘要 · Abstract (English)
Real-world datasets typically exhibit long-tailed (LT) distributions, where a few head classes dominate and many tail classes are severely underrepresented. While recent work shows that parameter-efficient fine-tuning (PEFT) methods like LoRA and AdaptFormer preserve tail-class performance on foundation models such as CLIP, we find that they do so at the cost of head-class accuracy. We identify the head-tail ratio, the proportion of head to tail classes, as a crucial but overlooked factor influencing this trade-off. Through controlled experiments on CIFAR100 with varying imbalance ratio ($ρ$) and head-tail ratio ($η$), we show that PEFT excels in tail-heavy scenarios but degrades in more balanced and head-heavy distributions. To overcome these limitations, we propose LT-Soups, a two-stage model soups framework designed to generalize across diverse LT regimes. In the first stage, LT-Soups averages models fine-tuned on balanced subsets to reduce head-class bias; in the second, it fine-tunes only the classifier on the full dataset to restore head-class accuracy. Experiments across six benchmark datasets show that LT-Soups achieves superior trade-offs compared to both PEFT and traditional model soups across a wide range of imbalance regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。