用视觉+对话帮用户定制饮食建议，体验更贴心。
Purrfessor: A Fine-tuned Multimodal LLaVA Diet Health Chatbot
- 用食物图像+营养数据微调LLaVA模型，支持图文互动
- 相比GPT-4，用户感知关怀度提升1.59倍，兴趣提升2.26倍
- 适合关注健康饮食、想提升交互体验的用户
本研究提出Purrfessor，一款通过多模态交互提供个性化饮食建议的AI聊天机器人。基于经过食物与营养数据微调的LLaVA模型，并结合人机协同设计，Purrfessor能分析用户上传的餐食图像并给出上下文相关建议。我们开展两项评估：(a) 模拟测试与人工验证，检验微调模型性能；(b) 一项2（用户身份：机器人 vs. 宠物）×3（模型：GPT-4 vs. LLaVA vs. 微调后的LLaVA）实验显示，相比GPT-4，Purrfessor显著提升了用户对关怀感（β=1.59, p=0.04）和兴趣度（β=2.26, p=0.01）的感知。用户访谈进一步强调响应速度、个性化和引导性对提升参与度的重要性。
原文摘要 · Abstract (English)
This study introduces Purrfessor, an innovative AI chatbot designed to provide personalized dietary guidance through interactive, multimodal engagement. Leveraging the Large Language-and-Vision Assistant (LLaVA) model fine-tuned with food and nutrition data and a human-in-the-loop approach, Purrfessor integrates visual meal analysis with contextual advice to enhance user experience and engagement. We conducted two studies to evaluate the chatbot's performance and user experience: (a) simulation assessments and human validation were conducted to examine the performance of the fine-tuned model; (b) a 2 (Profile: Bot vs. Pet) by 3 (Model: GPT-4 vs. LLaVA vs. Fine-tuned LLaVA) experiment revealed that Purrfessor significantly enhanced users' perceptions of care ($β= 1.59$, $p = 0.04$) and interest ($β= 2.26$, $p = 0.01$) compared to the GPT-4 bot. Additionally, user interviews highlighted the importance of interaction design details, emphasizing the need for responsiveness, personalization, and guidance to improve user engagement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。