对比LLM与传统模型在信贷风险预测中的表现与解释一致性
Interpreting LLMs as Credit Risk Classifiers: Do Their Feature Explanations Align with Classical ML?
- 用零样本提示让LLM和LightGBM做贷款违约预测
- LLM识别关键风险指标但特征重要性与传统模型差异大
- 自生成解释常不符实际特征贡献，需人工审核
大型语言模型（LLMs）通过零样本提示被探索用于分类任务，但其在结构化表格数据上的适用性，尤其是在高风险金融场景如信用风险评估中，仍缺乏系统研究。本研究在真实世界贷款违约预测任务上，系统比较了零样本LLM分类器与LightGBM（一种先进的梯度提升模型）的性能。通过评估预测效果、使用SHAP分析特征归因，并检验LLM自生成解释的可靠性，发现尽管LLM能识别关键金融风险指标，但其特征重要性排序与LightGBM显著不同，且自解释内容常与实证的SHAP归因不一致。结果表明，LLM作为独立模型在结构化金融风险预测中存在局限，其自解释可信度存疑。研究强调在高风险金融环境中部署LLM时，需进行可解释性审计、与可解释模型建立基线对比，并引入人工监督。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly explored as flexible alternatives to classical machine learning models for classification tasks through zero-shot prompting. However, their suitability for structured tabular data remains underexplored, especially in high-stakes financial applications such as financial risk assessment. This study conducts a systematic comparison between zero-shot LLM-based classifiers and LightGBM, a state-of-the-art gradient-boosting model, on a real-world loan default prediction task. We evaluate their predictive performance, analyze feature attributions using SHAP, and assess the reliability of LLM-generated self-explanations. While LLMs are able to identify key financial risk indicators, their feature importance rankings diverge notably from LightGBM, and their self-explanations often fail to align with empirical SHAP attributions. These findings highlight the limitations of LLMs as standalone models for structured financial risk prediction and raise concerns about the trustworthiness of their self-generated explanations. Our results underscore the need for explainability audits, baseline comparisons with interpretable models, and human-in-the-loop oversight when deploying LLMs in risk-sensitive financial environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。