让连续提示生成可读任务描述，揭示模型内部逻辑
Eliciting Textual Descriptions from Representations of Continuous Prompts
- 通过推理时提取提示表征，生成任务文本描述
- 性能越高，描述越准确，能反映真实任务意图
- 可发现提示中的偏见特征，帮助调试模型缺陷
连续提示（soft prompts）是大语言模型中一种高效的参数高效微调方法，但因其不可解释性常受质疑。以往方法将单个提示词投影到词汇空间进行解释，但易产生任意或矛盾的文本，且仅逐词分析。本文提出新方法，在模型推理过程中从提示表征中提取文本描述。使用Patchscopes变体InSPEcT在多个任务上验证，结果表明该方法生成的描述通常准确，且随着任务性能提升，描述愈发忠实。进一步分析显示，InSPEcT可揭示连续提示中的偏见特征，其存在与模型偏差预测相关。该方法为连续提示提供有效可解释性方案，可用于检测并改进提示中的不良属性。
原文摘要 · Abstract (English)
Continuous prompts, or "soft prompts", are a widely-adopted parameter-efficient tuning strategy for large language models, but are often less favorable due to their opaque nature. Prior attempts to interpret continuous prompts relied on projecting individual prompt tokens onto the vocabulary space. However, this approach is problematic as performant prompts can yield arbitrary or contradictory text, and it interprets prompt tokens individually. In this work, we propose a new approach to interpret continuous prompts that elicits textual descriptions from their representations during model inference. Using a Patchscopes variant (Ghandeharioun et al., 2024) called InSPEcT over various tasks, we show our method often yields accurate task descriptions which become more faithful as task performance increases. Moreover, an elaborated version of InSPEcT reveals biased features in continuous prompts, whose presence correlates with biased model predictions. Providing an effective interpretability solution, InSPEcT can be leveraged to debug unwanted properties in continuous prompts and inform developers on ways to mitigate them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。