让模型看图识菜，还能认出没见过的新菜式。
SalientFusion: Context-Aware Compositional Zero-Shot Food Recognition
- 用视觉注意力过滤背景干扰，结合深度信息区分主食和配菜
- 在两个新数据集上准确率超越现有方法，通用性更强
- 适合研究零样本学习或食物识别的开发者
食物识别备受关注，但新菜频出亟需识别未见菜类的能力，推动了零样本食物学习（ZSFL）的发展。本文提出组合式零样本食物识别（CZSFR）任务，将菜系与食材自然对应为组合零样本学习中的属性与对象。然而，该任务面临三大挑战：(1) 背景冗余干扰模型学习有效特征；(2) 主食与配菜角色混淆导致误分类；(3) 单一属性语义偏差引发理解混淆。为此，我们提出SalientFusion方法，包含两个组件：SalientFormer通过去除背景冗余并利用深度特征解决角色混淆；DebiasAT通过将提示与视觉特征对齐，降低语义偏差。在自建基准数据集CZSFood-90和CZSFood-164上，SalientFusion取得当前最优性能，并在主流通用CZSL数据集上表现领先。代码已开源：https://github.com/Jiajun-RUC/SalientFusion。
原文摘要 · Abstract (English)
Food recognition has gained significant attention, but the rapid emergence of new dishes requires methods for recognizing unseen food categories, motivating Zero-Shot Food Learning (ZSFL). We propose the task of Compositional Zero-Shot Food Recognition (CZSFR), where cuisines and ingredients naturally align with attributes and objects in Compositional Zero-Shot learning (CZSL). However, CZSFR faces three challenges: (1) Redundant background information distracts models from learning meaningful food features, (2) Role confusion between staple and side dishes leads to misclassification, and (3) Semantic bias in a single attribute can lead to confusion of understanding. Therefore, we propose SalientFusion, a context-aware CZSFR method with two components: SalientFormer, which removes background redundancy and uses depth features to resolve role confusion; DebiasAT, which reduces the semantic bias by aligning prompts with visual features. Using our proposed benchmarks, CZSFood-90 and CZSFood-164, we show that SalientFusion achieves state-of-the-art results on these benchmarks and the most popular general datasets for the general CZSL. The code is avaliable at https://github.com/Jiajun-RUC/SalientFusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。