测试大模型分析食谱是否适合糖尿病患者，发现结合饮食指南推理的模型表现更优。
Investigating the Ability of Large Language Models to Analyze Recipes for Diabetes
- 用三类提示词融合糖尿病饮食指南，引导模型分析食谱
- 7607个食谱中，多数模型谨慎判断，避免误判风险
- Mistral-7B和Llama 70B在融合指南推理时表现最佳
已有研究评估了大语言模型（LLMs）在膳食规划中的能力，结果积极。这些模型可处理自然语言输入，并利用预训练阶段学到的知识生成餐食计划。本文探究了LLMs分析给定食谱是否适合糖尿病患者的能力。主要挑战包括：从医学来源检索相关糖尿病饮食指南，将食谱分解为食材与烹饪方式，并据此判断其适宜性。为此，我们采用三种提示策略：（i）直接查询提示，（ii）上下文引导提示，（iii）示例上下文提示，分别整合不同层次的糖尿病饮食指南。我们构建了一个基准数据集，包含7607个食谱，其中3807个适合糖尿病，3800个不适合。实验结果表明，大多数模型在判断食谱适宜性时表现出谨慎态度，以避免不良后果。能够结合饮食指南进行推理的模型表现更优。总体而言，Mistral-7B和Llama 70B优于其他模型。
原文摘要 · Abstract (English)
Several studies have evaluated the ability of Large Language Models (LLMs) for meal planning, yielding positive outcomes. These models can process natural language inputs and leverage learned knowledge from their pretraining to generate meal plans. In this work, we investigate the ability of LLMs to analyze the suitability of given recipes for diabetes. The primary challenge for LLMs is to retrieve relevant dietary guidelines for diabetes, decompose recipes into ingredients and cooking methods, and apply these guidelines to determine the recipe's suitability. To study these challenges, we employ three kinds of prompts namely, (i) Direct Query Prompt (ii) Context-Guided Prompt, and (iii) Exemplary Context Prompt that incorporate different levels of diabetes dietary guidelines from medical sources. We introduce a benchmark dataset curated for this investigation consisting of 7607 recipes that include 3807 recipes suitable for diabetes and 3800 recipes not suitable for diabetes. Our results demonstrate that most LLMs are cautious in predicting recipes as suitable to prevent detrimental outcomes. Further, the models that can reason using the dietary guidelines performed better in predicting the suitability of recipes for diabetes. Overall, Mistral-7B and Llama 70B showed superior performance to their counterparts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。