用博弈论方法让大模型的概率预测更准更透明
Interpretable Probability Estimation with LLMs via Shapley Reconstruction
- 用谢林值分解输入因素贡献，量化每项影响
- 在金融、医疗等多领域提升预测准确率
- 可视化因子作用过程，增强决策可信度
大型语言模型（LLMs）凭借其丰富的知识和推理能力，具备估算不确定事件概率的潜力，可应用于金融预测、预防性医疗等智能决策场景。然而，直接提示LLM进行概率估计面临输出噪声大、预测过程不透明的问题。本文提出PRISM：基于谢林值的概率重建框架，通过计算各输入因素的边际贡献来分解LLM的预测结果，并聚合生成校准后的最终估计。实验表明，PRISM在金融、医疗、农业等多个领域均优于直接提示及其他基线方法。除性能提升外，该框架提供可解释的预测流程：案例研究展示了各因素如何影响最终结果，有助于建立对基于LLM决策支持系统的信任。
原文摘要 · Abstract (English)
Large Language Models (LLMs) demonstrate potential to estimate the probability of uncertain events, by leveraging their extensive knowledge and reasoning capabilities. This ability can be applied to support intelligent decision-making across diverse fields, such as financial forecasting and preventive healthcare. However, directly prompting LLMs for probability estimation faces significant challenges: their outputs are often noisy, and the underlying predicting process is opaque. In this paper, we propose PRISM: Probability Reconstruction via Shapley Measures, a framework that brings transparency and precision to LLM-based probability estimation. PRISM decomposes an LLM's prediction by quantifying the marginal contribution of each input factor using Shapley values. These factor-level contributions are then aggregated to reconstruct a calibrated final estimate. In our experiments, we demonstrate PRISM improves predictive accuracy over direct prompting and other baselines, across multiple domains including finance, healthcare, and agriculture. Beyond performance, PRISM provides a transparent prediction pipeline: our case studies visualize how individual factors shape the final estimate, helping build trust in LLM-based decision support systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。