融合血糖数据与餐前照片，提升糖尿病饮食热量估算精度。
Multimodal Fusion of Glucose Monitoring and Food Imagery for Caloric Content Prediction
- 用注意力机制和卷积网络处理食物图像，结合血糖与微生物组数据建模。
- 在40人数据集上实现RMSRE 0.2544，比基线模型提升超50%。
- 适合糖尿病管理、智能健康设备研发者参考。
有效饮食监测对管理2型糖尿病至关重要，但准确估算热量摄入仍是重大挑战。尽管连续血糖监测仪（CGMs）提供有价值的生理数据，却常因个体差异和餐食特异性而难以捕捉完整营养信息。本文提出一种多模态深度学习框架，联合利用CGM时序数据、人群特征/微生物组数据及餐前食物图像，以提升热量估计。模型采用基于注意力的编码和卷积特征提取处理图像，对CGM与微生物组数据使用多层感知机，并通过后期融合策略进行联合推理。我们在一个包含超过40名参与者的数据集上评估方法，该数据集同步采集了CGM、人群特征/微生物组数据及带标准化热量标签的餐照。模型达到0.2544的均方根相对误差（RMSRE），优于基线模型超过50%。结果表明，多模态传感在改善慢性病管理的自动化饮食评估工具方面具有潜力。
原文摘要 · Abstract (English)
Effective dietary monitoring is critical for managing Type 2 diabetes, yet accurately estimating caloric intake remains a major challenge. While continuous glucose monitors (CGMs) offer valuable physiological data, they often fall short in capturing the full nutritional profile of meals due to inter-individual and meal-specific variability. In this work, we introduce a multimodal deep learning framework that jointly leverages CGM time-series data, Demographic/Microbiome, and pre-meal food images to enhance caloric estimation. Our model utilizes attention based encoding and a convolutional feature extraction for meal imagery, multi-layer perceptrons for CGM and Microbiome data followed by a late fusion strategy for joint reasoning. We evaluate our approach on a curated dataset of over 40 participants, incorporating synchronized CGM, Demographic and Microbiome data and meal photographs with standardized caloric labels. Our model achieves a Root Mean Squared Relative Error (RMSRE) of 0.2544, outperforming the baselines models by over 50%. These findings demonstrate the potential of multimodal sensing to improve automated dietary assessment tools for chronic disease management.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。