无需分组标签也能提升模型在隐蔽子群体上的鲁棒性
Robustness Beyond Known Groups with Low-rank Adaptation
- 通过识别错误集中的低维表示子空间，仅用低秩调整分类器输出
- 在五大数据集上均显著提升最差子群体的准确率
- 适合无标注子群体、追求高效鲁棒性的实际应用
深度学习模型在优化平均准确率时，常在某些子群体上出现系统性失效。现实中，这些受影响的子群体往往未被标注或未知，因此需要不依赖预先指定子群体的方法来提升鲁棒性。现有方法通常依赖分组标签进行训练或选择。本文提出低秩误差感知适配（LEIA），一种两阶段方法：通过识别表示空间中模型错误集中的低维子空间，仅对分类器逻辑值进行低秩调整，直接针对潜在失败模式，无需修改主干网络或使用分组标签。在五个真实数据集上，我们评估了三种设置下的组鲁棒性：（1）完全不知子群体相关性，（2）部分已知，（3）完全已知。在所有设置下，LEIA均持续提升最差子群体性能，且快速、参数高效，对超参数选择不敏感。
原文摘要 · Abstract (English)
Deep learning models trained to optimize average accuracy often exhibit systematic failures on particular subpopulations. In real world settings, the subpopulations most affected by such disparities are frequently unlabeled or unknown, thereby motivating the development of methods that are performant on sensitive subgroups without being pre-specified. However, existing group-robust methods typically assume prior knowledge of relevant subgroups, using group annotations for training or model selection. We propose Low-rank Error Informed Adaptation (LEIA), a simple two-stage method that improves group robustness by identifying a low-dimensional subspace in the representation space where model errors concentrate. LEIA restricts adaptation to this error-informed subspace via a low-rank adjustment to the classifier logits, directly targeting latent failure modes without modifying the backbone or requiring group labels. Using five real-world datasets, we analyze group robustness under three settings: (1) truly no knowledge of subgroup relevance, (2) partial knowledge of subgroup relevance, and (3) full knowledge of subgroup relevance. Across all settings, LEIA consistently improves worst-group performance while remaining fast, parameter-efficient, and robust to hyperparameter choice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。