用可解释的概念分解连续提示，让模型决策更透明。
Concept Based Continuous Prompts for Interpretable Text Classification
- 将连续提示分解为人类可读的概念组合
- 仅用少量概念即达到与原方法相当的性能
- 适合关注模型可解释性的NLP研究者
连续提示已广泛用于提升多种自然语言任务的表现，但其增强机制仍不清晰。以往研究依赖单个词解释连续提示，缺乏整体语义理解。受概念瓶颈模型启发，我们提出一种通过分解为可读概念来解释连续提示的框架。具体而言，为确保分解可行性，我们证明总存在对应的概念嵌入矩阵和系数矩阵替代提示嵌入矩阵。随后，利用GPT-4o生成概念池，并通过新型子模优化算法筛选出具有判别性和代表性的候选概念。实验表明，本框架仅使用少数概念即可达到与原始P-tuning及基于词的方法相当的效果，且结果更具合理性。代码已开源：https://github.com/qq31415926/CD。
原文摘要 · Abstract (English)
Continuous prompts have become widely adopted for augmenting performance across a wide range of natural language tasks. However, the underlying mechanism of this enhancement remains obscure. Previous studies rely on individual words for interpreting continuous prompts, which lacks comprehensive semantic understanding. Drawing inspiration from Concept Bottleneck Models, we propose a framework for interpreting continuous prompts by decomposing them into human-readable concepts. Specifically, to ensure the feasibility of the decomposition, we demonstrate that a corresponding concept embedding matrix and a coefficient matrix can always be found to replace the prompt embedding matrix. Then, we employ GPT-4o to generate a concept pool and choose potential candidate concepts that are discriminative and representative using a novel submodular optimization algorithm. Experiments demonstrate that our framework can achieve similar results as the original P-tuning and word-based approaches using only a few concepts while providing more plausible results. Our code is available at https://github.com/qq31415926/CD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。