让文本特征既准确又可解释,靠的是人工与大模型共同验证其独立性。
Interpretable Discriminative Text Representations via Agreement and Label Disentanglement

- 用对比文本对生成候选特征,通过多模型一致性筛选
- 在10个任务中保持强预测力,且特征更少泄露标签信息
- 适合需要透明、可审计模型的AI应用开发者
可解释的文本表征应具备可被独立审计者理解的语义坐标。现有方法或使用无意义嵌入方向,或虽赋予自然语言名称却无法保证定义可复现或与目标标签区分。本文提出可操作的可解释标准:每个特征需满足概念清晰性(独立标注者间经机会校正的同意率)与标签解耦性(不等价于目标标签)。我们实现该标准的LLM辅助特征发现(LFD)方法:从对比文本对中生成特征,用跨模型Cohen's κ筛选,再以残差预测增益选择。分析表明κ筛选等价于每特征注释噪声上限。在7个语料库的10个文本分类任务中,LFD性能媲美强基线,但特征更清晰、更少标签泄露。232名审评员评估显示,其特征的人类-人类与人类-大模型一致率更高,且被普遍认为更少标签泄漏。结果表明,经一致性检验与标签解耦的坐标,可作为可解释文本分类的实用审计标准。
原文摘要 · Abstract (English)
Interpretable text representations should expose coordinates that are not only predictive, but also meaningful enough for independent auditors to apply. Existing discriminative representations often use anonymous embedding directions, while concept-bottleneck and LLM-assisted methods attach natural-language names to features without ensuring that those definitions are reproducible or distinct from the target label. We propose an operational criterion for interpretable discriminative text representations: each coordinate should satisfy conceptual clarity, measured by chance-adjusted agreement between independent annotators applying the feature definition, and label disentanglement, meaning the feature should not merely paraphrase the prediction target. We instantiate this criterion in LLM-assisted Feature Discovery (LFD), an iterative method that proposes lexical and semantic features from contrastive outcome-opposed text pairs, screens candidates using cross-LLM Cohen's $κ$, and selects features by residual held-out predictive gain. A stylized analysis connects the $κ$ screen to a per-feature annotation-noise bound, formalizing agreement as a reliability check. Across ten text-classification tasks spanning seven corpora, LFD matches the predictive performance of a strong text bottleneck baseline while producing substantially clearer and less label-entangled features. Human audits with 232 raters show that LFD features achieve higher human--human and human--LLM agreement than baseline concepts, and raters consistently judge them as less label-leaking. These results suggest that agreement-tested, label-disentangled coordinates provide a practical auditability standard for interpretable text classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。