用人类注意力指导图像混剪，提升模型训练效果
TdAttenMix: Top-Down Attention Guided Mixup
- 引入上下文注意力模块，融合高层语义与底层线索生成人工注意力
- 在8个基准上优于当前最优混剪方法，准确率显著提升
- 结合人眼注视数据评估图像标签一致性，适合视觉模型优化研究者
CutMix是一种数据增强策略，通过裁剪并粘贴图像块来混合训练数据。现有方法通常选择随机或显著区域，但这些区域常与真实标签不一致，误导模型训练。本文首次将人类注视行为引入CutMix，利用人类注意力同时受高层认知和低层线索驱动的特点，提出可控的自上而下注意力引导模块，生成兼顾上下文与局部细节的通用人工注意力。基于此，提出的TdAttenMix方法选择与当前标签相关区域进行裁剪，并动态调整标签混合比例。实验表明，TdAttenMix在八个不同基准上均超越现有最优Mixup方法。此外，我们设计了一种基于人眼注视的新度量指标，用于分析图像-标签不一致问题。项目页面： https://github.com/morning12138/TdAttenMix
原文摘要 · Abstract (English)
CutMix is a data augmentation strategy that cuts and pastes image patches to mixup training data. Existing methods pick either random or salient areas which are often inconsistent to labels, thus misguiding the training model. By our knowledge, we integrate human gaze to guide cutmix for the first time. Since human attention is driven by both high-level recognition and low-level clues, we propose a controllable Top-down Attention Guided Module to obtain a general artificial attention which balances top-down and bottom-up attention. The proposed TdATttenMix then picks the patches and adjust the label mixing ratio that focuses on regions relevant to the current label. Experimental results demonstrate that our TdAttenMix outperforms existing state-of-the-art mixup methods across eight different benchmarks. Additionally, we introduce a new metric based on the human gaze and use this metric to investigate the issue of image-label inconsistency. Project page: \url{https://github.com/morning12138/TdAttenMix}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。