用AR+AI帮你识别食材、推荐食谱，提升厨房效率。
ARChef: An iOS-Based Augmented Reality Cooking Assistant Powered by Multimodal Gemini LLM
- 通过手机摄像头实时识别食材，结合Gemini LLM生成推荐
- 支持用户输入饮食偏好，生成带营养信息的个性化菜单
- 基于ARKit构建iOS端交互界面，易用性好适合日常使用
烹饪对许多人来说具有挑战性，常导致依赖传统菜谱而出现遗漏食材、营养失衡和菜品不佳等问题。尽管增强现实（AR）可缓解这些问题，但现有应用普遍存在界面差、访问难等缺陷。本文提出一个基于iOS的AR烹饪助手原型，将计算机视觉（CV）与AR技术融合于烹饪流程中。利用Google Gemini大语言模型（LLM）识别摄像头视野内的食材，并生成包含详细营养信息的食谱建议。同时，借助Apple ARKit构建适配iOS设备的AR交互界面。用户可通过输入饮食偏好并评分，实现个性化推荐。通过三轮用户体验调研评估应用效果。该系统推动了可及性烹饪辅助技术的发展，旨在减少食物浪费，优化餐食规划体验。
原文摘要 · Abstract (English)
Cooking meals can be difficult, causing many to resort to cookbooks and online recipes. However, relying on these traditional methods of cooking often results in missing ingredients, nutritional hazards, and unsatisfactory meals. Using Augmented Reality (AR) can address these issues; however, current AR cooking applications have poor user interfaces and limited accessibility. This paper proposes a prototype of an iOS application that integrates AR and Computer Vision (CV) into the cooking process. We leverage Google's Gemini Large Language Model (LLM) to identify ingredients in the camera's field of vision and generate recipe choices with detailed nutritional information. Additionally, this application uses Apple's ARKit to create an AR user interface compatible with iOS devices. Users can personalize their meal suggestions by inputting their dietary preferences and rating each meal. The application's effectiveness is evaluated through three rounds of user experience surveys. This application advances the field of accessible cooking assistance technologies, aiming to reduce food wastage and improve the meal planning experience.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。