评测智能厨房助手的营养管理能力,填补动态环境下的持续状态追踪空白。
NutriBench-Kitchen: Benchmarking Embodied AI for Nutrition Management

- 构建多任务动态厨房基准,支持营养状态持续追踪
- 人类表现远超现有模型,尤其在长期状态与定量估算上
- 提出带分层记忆的诊断代理,验证显式状态表示的价值
具身厨房助手不仅需识别孤立图像中的食物,还需随时间追踪食材状态,并融合视觉观测、食谱与营养知识以实现约束感知决策。我们将其定义为‘具身营养管理’:感知营养相关事件,维持持久的食品状态,并用于知识驱动的规划。现有基准仅评估静态食物理解或具身烹饪动作,无法衡量智能体在动态厨房中持续更新和使用营养相关状态的能力。为此,我们引入【NutriBench-Kitchen】,包含160段烹饪视频生成的1,500对人工验证的问答对,覆盖五大任务类型:食材引入、记忆管理、食谱查询、长期规划与短期规划,涵盖食物状态构建、维护、知识检索及跨时域决策。对专有与开源大视觉语言模型的评估显示,其与人类表现存在显著差距,尤其在定量食材估算、长期状态追踪及多约束推理方面。我们进一步提出【Nutri-Vgent】——一个具备独立情景记忆、食品状态记忆与食谱记忆的诊断型长视频代理,其稳定提升证明了显式状态表示与结构化记忆对营养管理的重要性。NutriBench-Kitchen与Nutri-Vgent共同构成研究动态厨房中持续状态追踪与知识驱动推理的测试平台。代码已公开于 https://github.com/V1ol1n/NutriBench-Kitchen。
原文摘要 · Abstract (English)
An embodied kitchen assistant must do more than recognize food in isolated frames. It must track ingredient states over time and integrate visual observations with recipe and nutritional knowledge to support constraint-aware decision-making. We formalize this capability as \emph{Embodied Nutrition Management}: perceiving nutrition-relevant events, maintaining a persistent food state, and using it for knowledge-grounded planning. Existing benchmarks evaluate static food understanding or embodied cooking actions, but do not measure whether an agent can continuously update and use nutrition-relevant states in dynamic kitchens. To fill this gap, we introduce \textbf{NutriBench-Kitchen}, a benchmark containing 1,500 manually verified question--answer pairs from 160 cooking videos. It covers five task families: Ingredient Entry, Memory Management, Recipe Query, Long-Term Planning, and Short-Term Planning, spanning food-state construction, maintenance, knowledge retrieval, and decision-making across different planning horizons. Evaluations of proprietary and open-source large vision-language models reveal a substantial gap from human performance, particularly in quantitative ingredient estimation, long-term state tracking, and reasoning under interacting constraints. We further introduce \textbf{Nutri-Vgent}, a diagnostic long-video agent with separate episodic, food-state, and recipe memories. Its consistent improvements demonstrate the value of explicit state representations and structured memory for nutrition management. Together, NutriBench-Kitchen and Nutri-Vgent provide a testbed for studying persistent state tracking and knowledge-grounded reasoning in dynamic kitchens. Code is available at https://github.com/V1ol1n/NutriBench-Kitchen.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。