通过捕捉多方面内在关联，提升无监督推理片段提取效果。
MARE: Multi-Aspect Rationale Extractor on Unsupervised Rationale Extraction
- 用硬删除机制设计多方面多头注意力，同时编码多个文本块。
- 在文本前添加专属特殊标记，分别对应不同解释方面。
- 多任务训练降低开销，实测在两个数据集上达顶尖性能。
无监督推理片段提取旨在不依赖人工标注的情况下，从文本中提取支持模型预测的片段。以往方法常独立编码各个方面，难以捕捉其内在关联。本文提出多方面推理提取器（MARE），通过硬删除构建多方面多头注意力（MAMHA）机制，同时编码多个文本块，并为每个方面预先添加专属特殊标记。采用多任务训练以减少计算开销。在两个无监督推理提取基准上的实验表明，MARE 达到当前最优性能。消融实验进一步验证了方法的有效性。代码已开源：https://github.com/CSU-NLP-Group/MARE。
原文摘要 · Abstract (English)
Unsupervised rationale extraction aims to extract text snippets to support model predictions without explicit rationale annotation. Researchers have made many efforts to solve this task. Previous works often encode each aspect independently, which may limit their ability to capture meaningful internal correlations between aspects. While there has been significant work on mitigating spurious correlations, our approach focuses on leveraging the beneficial internal correlations to improve multi-aspect rationale extraction. In this paper, we propose a Multi-Aspect Rationale Extractor (MARE) to explain and predict multiple aspects simultaneously. Concretely, we propose a Multi-Aspect Multi-Head Attention (MAMHA) mechanism based on hard deletion to encode multiple text chunks simultaneously. Furthermore, multiple special tokens are prepended in front of the text with each corresponding to one certain aspect. Finally, multi-task training is deployed to reduce the training overhead. Experimental results on two unsupervised rationale extraction benchmarks show that MARE achieves state-of-the-art performance. Ablation studies further demonstrate the effectiveness of our method. Our codes have been available at https://github.com/CSU-NLP-Group/MARE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。