提升跨语言零样本学习的置信度校准，让模型更懂自己有多确定。
N2C2: Nearest Neighbor Enhanced Confidence Calibration for Cross-Lingual In-Context Learning
- 用最近邻缓存池融合支持样本预测，动态调整置信度。
- 在双语情感分类数据集上，准确率与校准误差均优于现有方法。
- 适合需要可靠置信度输出的跨语言应用，如多语种客服系统。
近期研究表明，提供示例可显著提升语言模型的上下文学习（ICL）性能。然而，跨语言场景下对ICL的模型校准与预测置信度关注不足。为此，我们对跨语言情感分类中的ICL进行了深入分析，发现其表现不佳，准确率低且校准误差高。针对此问题，我们提出N2C2方法，通过引入最近邻增强的分类器进行置信度校准。N2C2利用缓存的少样本实例数据存储池，缩小预测差距。具体而言，该方法结合数据存储池的预测结果，引入置信度感知分布、语义一致的检索表示及自适应邻居组合模块，有效利用有限的支持样本。在两个多语言情感分类数据集上的评估表明，N2C2优于传统ICL，在准确率和校准误差方面超越微调、提示调优及最新先进方法。
原文摘要 · Abstract (English)
Recent advancements of in-context learning (ICL) show language models can significantly improve their performance when demonstrations are provided. However, little attention has been paid to model calibration and prediction confidence of ICL in cross-lingual scenarios. To bridge this gap, we conduct a thorough analysis of ICL for cross-lingual sentiment classification. Our findings suggest that ICL performs poorly in cross-lingual scenarios, exhibiting low accuracy and presenting high calibration errors. In response, we propose a novel approach, N2C2, which employs a -nearest neighbors augmented classifier for prediction confidence calibration. N2C2 narrows the prediction gap by leveraging a datastore of cached few-shot instances. Specifically, N2C2 integrates the predictions from the datastore and incorporates confidence-aware distribution, semantically consistent retrieval representation, and adaptive neighbor combination modules to effectively utilize the limited number of supporting instances. Evaluation on two multilingual sentiment classification datasets demonstrates that N2C2 outperforms traditional ICL. It surpasses fine tuning, prompt tuning and recent state-of-the-art methods in terms of accuracy and calibration errors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。