提出24096张真实餐食图像的分割基准,解决重叠与长尾难题。
DishSeg24k: A Large-Scale Benchmark for Food Segmentation with Stochastic Expert Decoding

- 将解码过程建模为马尔可夫决策过程,分步探索边界不确定性
- 引入强化学习引导的专家混合模块,提升长尾类别下的模型表现
- 在新基准上实现mIoU、mDice、mAcc均超现有方法3%以上
食物分割对智能餐饮、膳食评估与推荐至关重要,但现有数据集难以反映真实用餐场景的复杂性。密集的盘间重叠、细粒度类别相似性以及极端长尾分布超出了当前数据集的刻画能力。为此,我们提出 extbf{DishSeg24k},一个包含24,096张图像、112,281个实例和278个细粒度类别的大规模盘级分割基准,覆盖真实就餐环境。基于此,我们进一步提出 extbf{Food Expert-Adaptive Segmentation Transformers (FEAST)},将基于查询的解码建模为马尔可夫决策过程(MDP),每个解码层更新视为逐步决策,沿盘边界探索不确定性。我们还设计了强化学习引导的专家混合(MoE)模块,采用双批评分离优化机制,分别处理任务导向的查询精炼与结构感知的专家路由。该设计促进专家专业化,缓解长尾分布下的专家退化问题。在DishSeg24k上的大量实验表明,FEAST在性能上达到新高度,相比此前方法分别提升+3.21\\% mIoU、+3.68\\" mDice 和 +4.00\\" mAcc。我们在FoodSeg103上也验证了其有效性。数据集与代码将公开发布。
原文摘要 · Abstract (English)
Food segmentation is essential for applications such as intelligent catering, dietary assessment, and recommendation. However, existing benchmarks fail to capture the complexity of real-world dining scenes. The challenges of dense inter-dish overlap, fine-grained class similarity, and extreme long-tail class distributions exceed the fidelity of current datasets. To fill this gap, we introduce \textbf{DishSeg24k}, a large-scale dish-level segmentation benchmark with 24,096 images, 112,281 instances, and 278 fine-grained categories in real-world dining environments. Based on DishSeg24k, we further propose \textbf{Food Expert-Adaptive Segmentation Transformers (FEAST)} to address these challenges. FEAST models query-based decoding as a Markov Decision Process (MDP), where each decoder layer update is treated as a sequential decision step that explores uncertainty along dish boundaries. We further redesign the decoder with a reinforcement learning (RL)-guided Mixture-of-Experts (MoE) module, in which a dual-critic decoupled optimization scheme separates task-oriented query refinement from structure-aware expert routing. This design promotes expert specialization and prevents expert collapse under long-tail category distributions. Finally, extensive experiments on DishSeg24k demonstrate the state-of-the-art performance of FEAST, which outperforms previous methods by {+3.21\%} mIoU, {+3.68\%} mDice, and {+4.00\%} mAcc, respectively. We further validate the effectiveness of FEAST on FoodSeg103. The dataset and code will be publicly released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。