arXiv:2511.08215cs.CVcs.LG2025-11

用EfficientNet-B4+Gemini分析食物图像,发现视觉识别是生成质量的瓶颈。

Evaluating Gemini LLM in Food Image-Based Recipe and Nutrition Description with EfficientNet-B4 Visual Backbone

  • 用EfficientNet-B4做视觉特征提取,Gemini生成食谱与营养信息。
  • 模型在自建中文食物数据集上达89.0%准确率,生成内容事实准确率9.2/10。
  • 发现分类错误会引发语义误差传播,适合多模态饮食应用研究者参考。

数字饮食应用的兴起催生了自动营养分析与烹饪指导的需求。本文对一种解耦的多模态食物识别流程进行综合对比评估,整合专用视觉骨干网络(EfficientNet-B4)与强大生成型大语言模型(Google Gemini LLM)。核心目标是评估视觉分类精度、模型效率与生成输出质量(营养数据与食谱)之间的权衡。我们将其与替代视觉骨干(VGG-16、ResNet-50、YOLOv8)及轻量级LLM(Gemma)进行基准测试。提出“语义误差传播”(SEP)形式化框架,分析视觉模块分类错误如何传递至生成结果。分析基于新构建的中文食物数据集CCFD,以缓解公共数据集中的文化偏差。实验表明,尽管EfficientNet-B4(Top-1准确率89.0%)在精度与效率间取得最佳平衡,Gemini(事实准确率9.2/10)生成质量更优,但系统整体效用仍受视觉前端感知精度的根本制约。通过逐类别分析,发现高语义相似性是最关键的失败模式。

原文摘要 · Abstract (English)

The proliferation of digital food applications necessitates robust methods for automated nutritional analysis and culinary guidance. This paper presents a comprehensive comparative evaluation of a decoupled, multimodal pipeline for food recognition. We evaluate a system integrating a specialized visual backbone (EfficientNet-B4) with a powerful generative large language model (Google's Gemini LLM). The core objective is to evaluate the trade-offs between visual classification accuracy, model efficiency, and the quality of generative output (nutritional data and recipes). We benchmark this pipeline against alternative vision backbones (VGG-16, ResNet-50, YOLOv8) and a lightweight LLM (Gemma). We introduce a formalization for "Semantic Error Propagation" (SEP) to analyze how classification inaccuracies from the visual module cascade into the generative output. Our analysis is grounded in a new Custom Chinese Food Dataset (CCFD) developed to address cultural bias in public datasets. Experimental results demonstrate that while EfficientNet-B4 (89.0\% Top-1 Acc.) provides the best balance of accuracy and efficiency, and Gemini (9.2/10 Factual Accuracy) provides superior generative quality, the system's overall utility is fundamentally bottlenecked by the visual front-end's perceptive accuracy. We conduct a detailed per-class analysis, identifying high semantic similarity as the most critical failure mode.

多模态食物识别生成质量视觉模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。