提出无监督方法检测并消除Transformer中的捷径学习问题。
Efficient Unsupervised Shortcut Learning Detection and Mitigation in Transformers
- 基于最新ML进展，构建无需标注的检测与缓解框架。
- 显著提升最差群体准确率和平均准确率，降低误判。
- 结果对专家有意义且可在普通电脑运行，效率高。
捷径学习（模型依赖无关特征）是机器学习应用中的重大挑战，尤其在医疗诊断等敏感决策场景中影响深远。本文利用最新机器学习进展，提出一种无监督框架，可同时检测与缓解Transformer中的捷径学习问题。在多个数据集上验证，结果表明该框架显著提升最差群体准确率（因捷径导致的误分类样本减少）与平均准确率，同时极大减少人工标注需求。此外，检测到的捷径具有实际意义，对人类专家有价值；框架计算高效，可在消费级硬件上运行。
原文摘要 · Abstract (English)
Shortcut learning, i.e., a model's reliance on undesired features not directly relevant to the task, is a major challenge that severely limits the applications of machine learning algorithms, particularly when deploying them to assist in making sensitive decisions, such as in medical diagnostics. In this work, we leverage recent advancements in machine learning to create an unsupervised framework that is capable of both detecting and mitigating shortcut learning in transformers. We validate our method on multiple datasets. Results demonstrate that our framework significantly improves both worst-group accuracy (samples misclassified due to shortcuts) and average accuracy, while minimizing human annotation effort. Moreover, we demonstrate that the detected shortcuts are meaningful and informative to human experts, and that our framework is computationally efficient, allowing it to be run on consumer hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。