用可能性分布指导分类,通过KL投影实现更合理的概率学习。
Probabilistic classification from possibilistic data: computing Kullback-Leibler projection with a possibility distribution
- 从可能性分布构建满足约束的可行概率集
- 使用KL投影使模型输出符合可能性结构,提升预测性能
- 适用于不确定性标注的分类任务,尤其适合自然语言推理
本文研究基于可能性监督的多分类学习。每个训练样本的标签是一个归一化可能性分布,表示各类别的相对可信度。基于该分布,构建一个非空闭凸的概率分布集合:需同时满足概率相容性与线性形状约束,确保相同可能性的类别概率相等,更高可能性的类别必然有更高概率。对模型输出的严格正概率向量,计算其在该集合上的Kullback-Leibler投影,得到最接近且满足约束的概率分布。通过最小化预测与投影间的分歧来训练模型,量化了满足约束所需的最小调整。投影采用基于负熵Bregman投影的Dykstra算法,并给出各约束集的显式投影公式。在合成数据和真实自然语言推理任务(ChaosNLI数据集)上的实验表明,该投影算法高效可用,所提出的基于投影的学习目标能有效提升预测性能。
原文摘要 · Abstract (English)
We consider learning with possibilistic supervision for multi-class classification. For each training instance, the supervision is a normalized possibility distribution that expresses graded plausibility over the classes. From this possibility distribution, we construct a non-empty closed convex set of admissible probability distributions by combining two requirements: probabilistic compatibility with the possibility and necessity measures induced by the possibility distribution, and linear shape constraints that must be satisfied to preserve the qualitative structure of the possibility distribution. Thus, classes with the same possibility degree receive equal probabilities, and if a class has a strictly larger possibility degree than another class, then it receives a strictly larger probability. Given a strictly positive probability vector output by a model for an instance, we compute its Kullback-Leibler projection onto the admissible set. This projection yields the closest admissible probability distribution in Kullback-Leibler sense. We can then train the model by minimizing the divergence between the prediction and its projection, which quantifies the smallest adjustment needed to satisfy the induced dominance and shape constraints. The projection is computed with Dykstra's algorithm using Bregman projections associated with the negative entropy, and we provide explicit formulas for the projections onto each constraint set. Experiments conducted on synthetic data and on a real-world natural language inference task, based on the ChaosNLI dataset, show that the proposed projection algorithm is efficient enough for practical use, and that the resulting projection-based learning objective can improve predictive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。