用多任务学习让遥感模型自动解释自身预测,无需额外数据。
Can Multitask Learning Enhance Model Explainability?
- 将不同模态作为辅助任务目标,而非额外输入
- 在数据稀缺时仍保持性能,误差可由辅助任务解释
- 适合需要模型可解释性的遥感分析场景
遥感提供多种类型和格式的卫星数据。多模态学习网络虽能提升模型性能,但复杂性降低了可解释性。本研究探索如何通过多任务学习,利用模态内在信息实现模型行为的内生解释。具体而言,不引入额外输入,而是将某些模态作为主任务之外的预测目标。该方法依赖于卫星数据丰富的信息量,且模态在推理时无需重新采集。实验表明:(1) 数据稀缺时,辅助模态无需部署时收集;(2) 模型性能与多模态基线相当,部分任务表现更优;(3) 主任务的预测误差可通过辅助任务的行为进行解释。在三个数据集上验证了该方法在分割、分类和回归任务中的有效性。代码已公开于 git.opendfki.de/hiba.najjar/mtl_explainability/。
原文摘要 · Abstract (English)
Remote sensing provides satellite data in diverse types and formats. The usage of multimodal learning networks exploits this diversity to improve model performance, except that the complexity of such networks comes at the expense of their interpretability. In this study, we explore how modalities can be leveraged through multitask learning to intrinsically explain model behavior. In particular, instead of additional inputs, we use certain modalities as additional targets to be predicted along with the main task. The success of this approach relies on the rich information content of satellite data, which remains as input modalities. We show how this modeling context provides numerous benefits: (1) in case of data scarcity, the additional modalities do not need to be collected for model inference at deployment, (2) the model performance remains comparable to the multimodal baseline performance, and in some cases achieves better scores, (3) prediction errors in the main task can be explained via the model behavior in the auxiliary task(s). We demonstrate the efficiency of our approach on three datasets, including segmentation, classification, and regression tasks. Code available at git.opendfki.de/hiba.najjar/mtl_explainability/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。