arXiv:2508.03996cs.CV2025-08被引 2

对比不同预训练数据对2D食物营养估算效果,发现公开数据不如私有数据有效。

Investigating the Impact of Large-Scale Pre-training on Nutritional Content Estimation from 2D Images

  • 用ViT模型在ImageNet和COYO上预训练,再微调用于营养估算。
  • 在Nutrition5k数据集上,私有JFT-300M预训练模型误差最低,优于公开数据。
  • 意外发现大规模公共数据(COYO)表现反而不如ImageNet,说明数据相关性更重要。

从2D图像估算食物营养成分是一项重要但极具挑战的任务,尤其因食物摆放、光照差异及缺乏深度信息导致体积与质量推断困难。现有先进方法依赖专有数据集进行大规模预训练,影响可复现性。本文研究大规模预训练数据对基于2D图像的营养估算模型性能的影响。我们微调并评估在ImageNet和COYO两个公开大数据集上预训练的Vision Transformer(ViT)模型,与基线CNN模型(InceptionV2、ResNet-50)及在专有JFT-300M数据集上预训练的先进方法进行比较。实验基于Nutrition5k数据集,该数据集包含大量带高精度营养标注的真实食物图像。通过均方绝对误差(MAE)和平均绝对百分比误差(MAE%)评估,结果显示在JFT-300M上预训练的模型显著优于在公开数据集上预训练的模型。出乎意料的是,基于超大规模公共数据集COYO的模型表现反而不如ImageNet预训练模型,挑战了初始假设。分析表明,预训练数据的规模、领域相关性和数据质量对2D营养估算中的迁移学习至关重要。

原文摘要 · Abstract (English)

Estimating the nutritional content of food from images is a critical task with significant implications for health and dietary monitoring. This is challenging, especially when relying solely on 2D images, due to the variability in food presentation, lighting, and the inherent difficulty in inferring volume and mass without depth information. Furthermore, reproducibility in this domain is hampered by the reliance of state-of-the-art methods on proprietary datasets for large-scale pre-training. In this paper, we investigate the impact of large-scale pre-training datasets on the performance of deep learning models for nutritional estimation using only 2D images. We fine-tune and evaluate Vision Transformer (ViT) models pre-trained on two large public datasets, ImageNet and COYO, comparing their performance against baseline CNN models (InceptionV2 and ResNet-50) and a state-of-the-art method pre-trained on the proprietary JFT-300M dataset. We conduct extensive experiments on the Nutrition5k dataset, a large-scale collection of real-world food plates with high-precision nutritional annotations. Our evaluation using Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAE%) reveals that models pre-trained on JFT-300M significantly outperform those pre-trained on public datasets. Unexpectedly, the model pre-trained on the massive COYO dataset performs worse than the model pre-trained on ImageNet for this specific regression task, refuting our initial hypothesis. Our analysis provides quantitative evidence highlighting the critical role of pre-training dataset characteristics, including scale, domain relevance, and curation quality, for effective transfer learning in 2D nutritional estimation.

营养估算视觉预训练迁移学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。