提升AI的可信度判断能力，能更好辅助人类决策。
Beyond Accuracy: How AI Metacognitive Sensitivity improves AI-assisted Decision Making
- 用可信度区分对错预测，评估AI自我认知能力
- 低准确率但高可信度的AI反而更助人决策
- 适合需要人机协作的医疗、金融等场景
在人类依赖AI输入做决策的场景中，AI的预测准确率及其置信度估计的可靠性共同影响决策质量。本文提出‘AI元认知敏感性’概念——即AI能否准确通过置信度区分正确与错误预测，并构建理论框架分析其与预测准确率在人机协作中的联合影响。分析表明，在特定条件下，即使预测准确率较低，只要元认知敏感性更高，AI仍能提升人类整体决策准确率。行为实验证实，更高的元认知敏感性显著改善人类决策表现。研究强调，评估AI辅助系统不应仅看准确率，还应关注其置信度可靠性，优化二者才能实现更优决策结果。
原文摘要 · Abstract (English)
In settings where human decision-making relies on AI input, both the predictive accuracy of the AI system and the reliability of its confidence estimates influence decision quality. We highlight the role of AI metacognitive sensitivity -- its ability to assign confidence scores that accurately distinguish correct from incorrect predictions -- and introduce a theoretical framework for assessing the joint impact of AI's predictive accuracy and metacognitive sensitivity in hybrid decision-making settings. Our analysis identifies conditions under which an AI with lower predictive accuracy but higher metacognitive sensitivity can enhance the overall accuracy of human decision making. Finally, a behavioral experiment confirms that greater AI metacognitive sensitivity improves human decision performance. Together, these findings underscore the importance of evaluating AI assistance not only by accuracy but also by metacognitive sensitivity, and of optimizing both to achieve superior decision outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。