随机性让大模型解释不一致，需评估解释的稳定性。
Explanation sensitivity to the randomness of large language models: the case of journalistic text classification
- 用不同随机种子训练模型，准确率相似但解释差异大
- 基于文本特征的简单模型解释更稳定但准确率较低
- 引入大模型解释特征可提升简单模型表现
大语言模型在自然语言处理任务中表现优异，但其可解释性面临挑战。本文以法语观点性新闻文本分类任务为例，研究训练中的随机因素对模型解释的影响。使用微调后的CamemBERT模型与基于重要性传播的解释方法，发现不同随机种子训练出的模型虽有相近准确率，但解释结果差异显著。因此，需分析解释的统计分布以保障可解释性。进一步探索基于文本特征的简化模型，其解释稳定但准确率较低。通过引入来自CamemBERT解释的特征，该模型性能可得到提升。研究结果揭示了训练随机性导致解释敏感性的根源，为未来研究提供了新方向。
原文摘要 · Abstract (English)
Large language models (LLMs) perform very well in several natural language processing tasks but raise explainability challenges. In this paper, we examine the effect of random elements in the training of LLMs on the explainability of their predictions. We do so on a task of opinionated journalistic text classification in French. Using a fine-tuned CamemBERT model and an explanation method based on relevance propagation, we find that training with different random seeds produces models with similar accuracy but variable explanations. We therefore claim that characterizing the explanations' statistical distribution is needed for the explainability of LLMs. We then explore a simpler model based on textual features which offers stable explanations but is less accurate. Hence, this simpler model corresponds to a different tradeoff between accuracy and explainability. We show that it can be improved by inserting features derived from CamemBERT's explanations. We finally discuss new research directions suggested by our results, in particular regarding the origin of the sensitivity observed in the training randomness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。