轻量级食物识别新模型，通过自适应稀疏注意力提升效率与精度
Fraesormer: Learning Adaptive Sparse Transformer for Efficient Food Recognition
- 设计自适应稀疏注意力机制，动态筛选关键特征交互
- 实现多尺度特征融合，在多个数据集上超越现有方法
- 适合资源受限场景下的实时食物识别应用
近年来，Transformer 在食物识别中取得显著进展。然而，现有方法在轻量级食物识别中仍面临两大挑战：(1) 二次复杂度及与无关标记的冗余特征交互；(2) 固定特征识别与单尺度表示，忽视了食物图像的非结构化和非固定特性，以及对多尺度特征的需求。为此，我们提出一种自适应高效稀疏 Transformer 架构 Fraesormer，包含两项核心设计：自适应 Top-k 稀疏局部注意力（ATK-SPA）与层级尺度敏感特征门控网络（HSSFGN）。ATK-SPA 使用可学习的门控动态 Top-K 操作器（GDTKO）保留关键注意力得分，过滤低查询-键匹配，减少特征聚合障碍；同时引入部分通道机制，降低冗余并促进专家信息流动，实现局部-全局协同建模。HSSFGN 通过门控机制实现多尺度特征表示，增强上下文语义信息。大量实验表明，Fraesormer 在多个基准数据集上优于当前最优方法。代码已公开于 https://zs1314.github.io/Fraesormer。
原文摘要 · Abstract (English)
In recent years, Transformer has witnessed significant progress in food recognition. However, most existing approaches still face two critical challenges in lightweight food recognition: (1) the quadratic complexity and redundant feature representation from interactions with irrelevant tokens; (2) static feature recognition and single-scale representation, which overlook the unstructured, non-fixed nature of food images and the need for multi-scale features. To address these, we propose an adaptive and efficient sparse Transformer architecture (Fraesormer) with two core designs: Adaptive Top-k Sparse Partial Attention (ATK-SPA) and Hierarchical Scale-Sensitive Feature Gating Network (HSSFGN). ATK-SPA uses a learnable Gated Dynamic Top-K Operator (GDTKO) to retain critical attention scores, filtering low query-key matches that hinder feature aggregation. It also introduces a partial channel mechanism to reduce redundancy and promote expert information flow, enabling local-global collaborative modeling. HSSFGN employs gating mechanism to achieve multi-scale feature representation, enhancing contextual semantic information. Extensive experiments show that Fraesormer outperforms state-of-the-art methods. code is available at https://zs1314.github.io/Fraesormer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。