通过分层修正提升弱类别准确率，实现更公平的文本分类。
Ensemble Debiasing Across Class and Sample Levels for Fairer Prompting Accuracy
- 用阶梯函数构建集成去偏机制，灵活调整类和样本级概率。
- 在7个数据集上实现最优整体准确率，且各类别表现更均衡。
- 对小模型和大模型均有效，尤其适合医疗领域任务。
语言模型虽具备强大的少样本学习能力，在文本分类任务中表现良好，但其结果存在严重的类别准确率不平衡问题。我们认为，提升整体准确率不应依赖强化强类别，而应着力于提高弱类别表现。为此，提出基于赫维赛德阶跃函数的集成去偏方法，可在类级别和样本级别灵活修正上下文学习得到的类别概率。在七个文本分类基准测试中，使用 Llama-2-13B 进行评估，该方法实现了当前最优的整体准确率提升，并保持了类别间平衡。进一步分析表明，样本级修正对于提升弱类别至关重要。由于有效纠正弱类别，该方法在更大模型变体 Llama-2-70B 上也带来显著性能提升，尤其是在生物医学领域任务中，充分证明了双层级集成去偏的必要性。源代码已开源：https://github.com/NUS-HPC-AI-Lab/DCS。
原文摘要 · Abstract (English)
Language models are strong few-shot learners and achieve good overall accuracy in text classification tasks, masking the fact that their results suffer from great class accuracy imbalance. We believe that the pursuit of overall accuracy should not come from enriching the strong classes, but from raising up the weak ones. To address the imbalance, we propose a Heaviside step function based ensemble debiasing method, which enables flexible rectifications of in-context learned class probabilities at both class and sample levels. Evaluations with Llama-2-13B on seven text classification benchmarks show that our approach achieves state-of-the-art overall accuracy gains with balanced class accuracies. More importantly, we perform analyses on the resulted probability correction scheme, showing that sample-level corrections are necessary to elevate weak classes. Due to effectively correcting weak classes, our method also brings significant performance gains to a larger model variant, Llama-2-70B, especially on a biomedical domain task, further demonstrating the necessity of ensemble debiasing at both levels. Our source code is available at https://github.com/NUS-HPC-AI-Lab/DCS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。