通过模糊规则动态修正分类概率,显著提升大模型少样本学习的公平性。
Let the Fuzzy Rule Speak: Enhancing In-context Learning Debiasing with Interpretability
- 基于三角隶属函数构建模糊规则,按概率区间精准调整每条数据的分类结果
- 在7个基准数据集上将类别准确率偏差降低56%,整体准确率提升21%
- 无需重训练模型,可解释性强,适合需高公平性的实际应用
大型语言模型在使用上下文学习进行文本分类时,常出现类别准确率不平衡问题,导致部分类别预测不足,影响实用性和安全性。本文发现该问题源于某些类别在上下文学习中持续获得过高概率,且不同概率区间对失衡的影响程度不同,因此可实现针对性修正。提出FuRud(基于模糊规则优化的去偏方法),利用带有三角隶属函数的模糊集,根据每个样本的概率所属区间,选择最优的19种修正函数之一,通过求解非线性整数规划问题最小化类别准确率偏差(COBias)并最大化整体准确率。整个过程不更新模型,在推理阶段完成样本级概率修正。在7个基准数据集上,该方法使COBias下降56%,整体准确率相对提升21%,优于现有先进去偏方法。
原文摘要 · Abstract (English)
Large language models (LLMs) often struggle with balanced class accuracy in text classification tasks using in-context learning (ICL), hindering some practical uses due to user dissatisfaction or safety risks caused by misclassifications. Retraining LLMs to address root causes in data or model priors is neither easy nor cost-effective. This paper delves deeper into the class accuracy imbalance issue, identifying that it arises because certain classes consistently receive disproportionately high ICL probabilities, causing under-prediction and lower accuracy for others. More importantly, probability ranges affect the imbalance differently, allowing for precise, range-specific corrections. We introduce FuRud (Fuzzy Rule Optimization-based Debiasing), a method for sample-level class probability correction. FuRud tackles interpretability challenges by determining why certain classes need corrections and tailoring adjustments for each instance's class probabilities which is powered by fuzzy sets with triangular membership functions, transforming a class probability based on the range it belongs to. By solving a nonlinear integer programming problem with a labeled set of ICL class probabilities to minimize class accuracy bias (COBias) and maximize overall accuracy, each class selects an optimal correction function from 19 triangular membership functions without updating an LLM, and the selected functions correct test instances at inference. Across seven benchmark datasets, FuRud reduces COBias by over half (56%) and improves overall accuracy by 21% relatively, outperforming state-of-the-art debiasing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。