将食物图像按食材分像素分割，助力营养意识提升
Ingredient-Level Food Image Segmentation for Nutrition Awareness
- 用SegFormer模型实现像素级食材识别
- B1模型在测试集上达32.04%平均交并比
- 输出食材占比图，直观展示餐食构成
食物图像通常包含多个可见食材，仅用一个菜名标签会掩盖其重要视觉结构。本文在FoodSeg103数据集上研究食材级语义分割，模型为每个像素预测对应食材类别。采用两种SegFormer变体进行微调：小型基线模型SegFormer-B0和大型最终模型SegFormer-B1，均使用ImageNet预训练的MiT主干网络，并初始化104类输出层。在保留的FoodSeg103测试集（2,135张图像）上，B0模型取得0.7709像素准确率和0.2521平均交并比，B1模型分别达到0.7929和0.3204，各项指标均优于前者，其中平均交并比绝对提升0.0683。系统还将预测掩码转化为可见食材面积占比，生成餐食构成的可视化摘要。该摘要可作为初步营养意识提示，提供类似盘餐指导的视觉参考，但不直接估算卡路里、宏量营养素、食物质量、体积、密度或真实份量。
原文摘要 · Abstract (English)
Food images often contain several visible ingredients, so assigning one dish label to an entire image hides important visual structure. This work studies ingredient-level semantic segmentation on FoodSeg103, where the model predicts an ingredient class for each pixel. Two SegFormer variants were fine-tuned and evaluated under a controlled setup: SegFormer-B0 as the smaller baseline model and SegFormer-B1 as the larger final model. Both models use ImageNet-pretrained MiT backbones with newly initialized 104-class output layers. On the held-out FoodSeg103 test split of 2,135 images, B0 achieved 0.7709 pixel accuracy and 0.2521 mean IoU, while B1 achieved 0.7929 pixel accuracy and 0.3204 mean IoU. B1 improved every saved test metric, including a +0.0683 absolute gain in mean IoU. The system also converts predicted masks into visible ingredient-area percentages, giving a simple visual composition summary of the predicted meal. This summary can serve as a first-pass nutrition-awareness cue by providing a visual alternative to detailed food tracking similar to plate-based meal guidance, but it is not a direct estimate of calories, macronutrients, food mass, volume, density, or true portion size.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。