用大模型从文本生成可解释的少量特征,提升机器学习可解释性。
LLM-based feature generation from text for interpretable machine learning
- 用LLM从科学文本中提取62个语义清晰的可解释特征。
- 在两个数据集上预测影响力表现,效果媲美SciBERT(768维)。
- 生成的特征能直接对应方法严谨性等概念,适合需要透明决策的场景。
现有文本表示如嵌入向量和词袋模型因维度高且缺乏特征级可解释性,不适用于规则学习。本文探索大型语言模型(LLMs)能否从中提取少量可解释特征。我们在包含数千篇跨学科科学论文的CORD-19和M17+数据集上进行实验,目标为研究影响力的代理指标。基于与研究影响力显著相关的检验,结果显示LLama 2生成的特征具有语义意义。随后,我们使用这些特征在文本分类任务中预测CORD-19的引用率(二值变量)和M17+的专家评分等级(5分类有序变量)。基于LLM生成特征的机器学习模型表现与最先进的科学文本嵌入模型SciBERT相当。所用特征仅62个,远少于SciBERT的768维,且特征可直接解释,对应方法严谨性、新颖性或语法正确性等概念。最终,我们提取出少量高度可解释的动作规则。相同LLM特征集在主题差异较大的两个数据集中均取得一致竞争力结果,表明该方法具备跨领域泛化能力。
原文摘要 · Abstract (English)
Existing text representations such as embeddings and bag-of-words are not suitable for rule learning due to their high dimensionality and absent or questionable feature-level interpretability. This article explores whether large language models (LLMs) could address this by extracting a small number of interpretable features from text. We demonstrate this process on two datasets (CORD-19 and M17+) containing several thousand scientific articles from multiple disciplines and a target being a proxy for research impact. An evaluation based on testing for the statistically significant correlation with research impact has shown that LLama 2-generated features are semantically meaningful. We consequently used these generated features in text classification to predict the binary target variable representing the citation rate for the CORD-19 dataset and the ordinal 5-class target representing an expert-awarded grade in the M17+ dataset. Machine-learning models trained on the LLM-generated features provided similar predictive performance to the state-of-the-art embedding model SciBERT for scientific text. The LLM used only 62 features compared to 768 features in SciBERT embeddings, and these features were directly interpretable, corresponding to notions such as article methodological rigor, novelty, or grammatical correctness. As the final step, we extract a small number of well-interpretable action rules. Consistently competitive results obtained with the same LLM feature set across both thematically diverse datasets show that this approach generalizes across domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。