修复大模型零样本分类中的隐性信息丢失，提升判断准确性与可信度。
The Silent Vote: Improving Zero-Shot LLM Reliability by Aggregating Semantic Neighborhoods
- 通过聚合目标标签的语义邻域得分，恢复被忽略的潜在信息。
- 在多个指标上显著降低校准误差，提升分类性能与置信度可靠性。
- 适合需要高可信度输出的零样本推理场景，如情感分析与评论检测。
大语言模型在复杂推理任务中越来越多地作为零样本分类器使用。然而，标准约束解码存在我们定义为重归一化偏差的现象：当模型被限制在少数目标标签时，标准softmax会丢弃原始分布中语义同义词所占的概率质量。这种损失我们称为‘沉默投票’，导致模型产生人为的过度自信并出现校准不良。为此，我们提出语义softmax——一种推理时的层，通过聚合每个目标标签周围语义邻域的得分来恢复这部分丢失的信息。我们在Qwen-3和Phi-4-mini模型上，基于GoEmotions和Civil Comments数据集进行评估。结果表明，该方法在所有评测指标上均取得一致提升：显著降低预期校准误差（ECE）和Brier Score，同时提高AUROC与宏平均F1得分。通过考虑语言细微差别，该方法为零样本分类提供了更校准、更准确的替代方案。
原文摘要 · Abstract (English)
Large Language Models are increasingly used as zero-shot classifiers in complex reasoning tasks. However, standard constrained decoding suffers from a phenomenon we define as Renormalization Bias. When a model is restricted to a small set of target labels, the standard softmax operation discards the probability mass assigned to semantic synonyms in the original distribution. This loss of information, which we call the Silent Vote, results in artificial overconfidence and poor calibration. We propose Semantic Softmax, an inference-time layer that recovers this lost information by aggregating the scores of the semantic neighborhood surrounding each target label. We evaluate this approach on Qwen-3 and Phi-4-mini models using GoEmotions and Civil Comments datasets. Our results demonstrate consistent improvements across all evaluation metrics: Semantic Softmax substantially reduces Expected Calibration Error (ECE) and Brier Score, while simultaneously enhancing discriminative performance in terms of AUROC and Macro-F1. By accounting for linguistic nuances, our method provides a more calibrated and accurate alternative for zero-shot classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。