让Transformer同时分类并给出每段文字的相关性评分
Rationalizing Transformer Predictions via End-To-End Differentiable Self-Training
- 用单模型替代三个角色，训练更稳定高效
- 无需人工标注，与人类判断对齐度达当前最优
- 可生成每个类别的解释，适合需要可解释性的场景
我们提出一种端到端可微的训练范式，用于稳定训练可解释的Transformer分类器。该方法使单一模型同时完成样本分类和输入词元相关性评分。基于广泛采用的三玩家博弈训练框架，传统方法需分别训练理由选择器、分类器和补全分类器。本文通过让单个模型承担全部三个角色，简化了流程，提升了训练稳定性，并避免了现有方法常见的训练不收敛问题。进一步地，该范式可生成类别级解释，结合最新的参数化与正则化技术，实现了显著提升的可解释性，且在无显式监督下达到与人类标注一致性的最先进水平。
原文摘要 · Abstract (English)
We propose an end-to-end differentiable training paradigm for stable training of a rationalized transformer classifier. Our approach results in a single model that simultaneously classifies a sample and scores input tokens based on their relevance to the classification. To this end, we build on the widely-used three-player-game for training rationalized models, which typically relies on training a rationale selector, a classifier and a complement classifier. We simplify this approach by making a single model fulfill all three roles, leading to a more efficient training paradigm that is not susceptible to the common training instabilities that plague existing approaches. Further, we extend this paradigm to produce class-wise rationales while incorporating recent advances in parameterizing and regularizing the resulting rationales, thus leading to substantially improved and state-of-the-art alignment with human annotations without any explicit supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。