用多模态AI让饮食记录更灵活准确，提升用户持续使用意愿。
SnappyMeal: Design and Longitudinal Evaluation of a Multimodal AI Food Logging Application
- 结合图像、文本和收据信息自动补全饮食细节
- 3周真实场景测试中记录超500次食物，用户感知准确率高
- 适合健康追踪、营养管理人群，尤其适合不愿手记者
饮食记录在揭示饮食与健康、医疗、健身等关系中至关重要。通过与营养专家及饮食追踪者访谈发现，当前手写或应用记录方式僵化，导致依从性低且营养数据可能不准确。为此，我们提出SnappyMeal——一个基于多模态输入的AI饮食记录系统。该系统通过目标驱动的追问机制智能获取用户缺失信息，并利用用户购物收据和营养数据库进行信息检索以提升准确性。我们在公开营养基准和为期三周的多用户真实环境部署中评估了该系统，共收集超过500条食物记录。用户高度认可多种输入方式，普遍认为记录结果准确。结果表明，多模态AI系统可显著提升饮食记录的灵活性与上下文感知能力，为智能自我追踪应用奠定基础。
原文摘要 · Abstract (English)
Food logging, both self-directed and prescribed, plays a critical role in uncovering correlations between diet, medical, fitness, and health outcomes. Through conversations with nutritional experts and individuals who practice dietary tracking, we find current logging methods, such as handwritten and app-based journaling, are inflexible and result in low adherence and potentially inaccurate nutritional summaries. These findings, corroborated by prior literature, emphasize the urgent need for improved food logging methods. In response, we propose SnappyMeal, an AI-powered dietary tracking system that leverages multimodal inputs to enable users to more flexibly log their food intake. SnappyMeal introduces goal-dependent follow-up questions to intelligently seek missing context from the user and information retrieval from user grocery receipts and nutritional databases to improve accuracy. We evaluate SnappyMeal through publicly available nutrition benchmarks and a multi-user, 3-week, in-the-wild deployment capturing over 500 logged food instances. Users strongly praised the multiple available input methods and reported a strong perceived accuracy. These insights suggest that multimodal AI systems can be leveraged to significantly improve dietary tracking flexibility and context-awareness, laying the groundwork for a new class of intelligent self-tracking applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。