发现大模型的自我解释真实性与分类准确率相关，可用来评估预测可信度。
Exploring the Reliability of Self-explanation and its Relationship with Classification in Language Model-driven Financial Analysis
- 通过事实性和因果性量化评估大模型的自我解释质量。
- 实证发现解释质量与分类准确率存在显著正相关关系。
- 适合关注模型可信度与金融决策解释性的研究者参考。
语言模型在推理和深度财务分析中展现出卓越能力，但以往研究多关注分类性能而忽视可解释性，或预设精细解释即对应更高准确率。本文基于公开金融数据集，定量评估了语言模型自我解释的事实性和因果性。结果表明，分类准确率与自我解释的事实性、因果性之间存在统计显著关系。该研究为通过自我解释近似分类置信度、并利用专属推理优化分类提供了实证基础。
原文摘要 · Abstract (English)
Language models (LMs) have exhibited exceptional versatility in reasoning and in-depth financial analysis through their proprietary information processing capabilities. Previous research focused on evaluating classification performance while often overlooking explainability or pre-conceived that refined explanation corresponds to higher classification accuracy. Using a public dataset in finance domain, we quantitatively evaluated self-explanations by LMs, focusing on their factuality and causality. We identified the statistically significant relationship between the accuracy of classifications and the factuality or causality of self-explanations. Our study built an empirical foundation for approximating classification confidence through self-explanations and for optimizing classification via proprietary reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。