揭示情绪识别准确率天花板受标注差异影响，提出新方法量化不可减少误差。
Bias-Corrected Ceilings of Emotion Predictability from Human Label Variation Based on Instance-Level Fano Bounds
- 基于人类共识分布估计，用贝叶斯方法校正标注偏差
- 发现至少33%的情绪分类误差无法通过改进模型消除
- 适用于评估情绪识别研究是否接近真实上限
文本情绪识别在基准测试上持续进步，但其准确率上限是否已达却很少被严谨追问。本文不试图给出单一上限数值，而是量化标注有限性、估计器选择、标注噪声和评估协议对上限的影响，从而规范对性能饱和的宣称。我们提出偏差校正情绪上限估计（BACE）框架，可估计校正偏差后的上限，区分不可减少与可减少误差，并以严格门控机制避免循环论证。采用锚定狄利克雷混合经验贝叶斯估计器，在插值估计与NSB之间恢复人类共识分布；通过标注者拆分、噪声去卷积和固定声明门控，实现无循环的误差归因。方法上，无约束点估计显示可达范围从0.38到1.03，表明单一估计器无法判定饱和。实质上，唯一通过声明门控的结论是：在GoEmotions数据集上，代表性分类器至少33%的误差为不可减少，该模式在攻击性与讽刺识别中同样成立。
原文摘要 · Abstract (English)
Emotion recognition from text keeps improving on benchmarks, yet whether an accuracy ceiling has been reached is seldom asked with discipline. Our aim is not to pin this ceiling to a single number, but to quantify how far it depends on finite annotation, estimator choice, annotation noise, and the evaluation protocol, and thereby to discipline how confidently saturation can be claimed. We propose Bias-corrected Affective Ceiling Estimation (BACE), an analysis framework that estimates a bias-corrected ceiling, separates irreducible from reducible error, and disciplines the resulting claims. An anchored Dirichlet-mixture empirical Bayes estimator, bracketed between plug-in and NSB, recovers the human-consensus distribution; an annotator split, a noise deconvolution, and a fixed claim gate then attribute error without circularity. Methodologically, unconstrained point estimates place reachability anywhere from 0.38 to 1.03, so saturation cannot be decided by any single estimator. Substantively, the only assertion passing the claim gate is that at least about 33% of a representative classifier's error on GoEmotions is irreducible, with the same pattern recurring on offensiveness and irony.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。