用图结构显式表达烹饪过程,让AI生成更符合真实流程的食谱。
ReGraph: Learning to Generate Recipe Graphs from Food Images

- 将食材、动作、工具构建成带状态属性的实体图,明确烹饪步骤关系。
- 现有模型生成的食谱虽文字流畅,但实际流程结构匹配度很低。
- 新框架RGL显著提升实体和流程关系生成,尤其适合研究过程推理的学者。
近期大型多模态模型在从食物图像生成食谱方面取得了显著进展。然而,烹饪是一个有序的动作序列,食材经历状态变化,而自由文本食谱往往隐含且混淆了实体、中间状态与依赖关系。图结构能显式表达这种过程性知识,为评估模型是否真正理解烹饪流程提供依据。为此,我们构建了大规模食谱图数据集ReGraph,将食材、烹饪动作、工具作为实体,用属性描述食材状态变化,并通过类型化关系编码操作目标、目的地及流程顺序。ReGraph还引入显式的食谱推理链(RR-CoT),辅助监督过程分解与结构化图生成。基于此,我们提出配方图学习(RGL)框架,采用两阶段策略,使大模型能从食物图像生成细粒度的结构化食谱图。在确定性的模式匹配协议下,实验显示:现有方法虽在文本生成质量上表现优异,但在与ReGraph模式对齐的实体与关系结构上表现有限;相比之下,RGL在两种代表性大模型基础上均显著提升了实体与流程关系生成能力,分析进一步表明,细粒度食材状态捕捉仍是最大挑战。
原文摘要 · Abstract (English)
Recent Large Multimodal Models (LMMs) have achieved impressive performance in recipe generation from food images.However, cooking is a structured transformation process in which ingredients undergo state changes through ordered actions,while free-form recipe language leaves the corresponding entities, intermediate states, and dependencies largely implicit and entangled.A graph representation makes this procedural knowledge explicit and compositional, providing a structured basis for assessing whether model outputs encode process-level knowledge rather than merely presenting plausible textual descriptions. To address this limitation, we present ReGraph, a large-scale recipe graph dataset that represents ingredients, cooking actions, and tools as entities, uses entity attributes to describe ingredient state changes, and employs typed relations to encode manipulation targets, destinations, and procedural ordering. ReGraph further incorporates explicit Recipe Reasoning Chain-of-Thought (RR-CoT) traces, providing auxiliary supervision for procedural decomposition and structured graph generation. Building on ReGraph, we propose Recipe Graph Learning (RGL), a two-stage framework that enables LMMs to generate a plausible fine-grained cooking workflow from a food image in the form of a structured recipe graph. Under a deterministic, schema-aware matching protocol, our experiments reveal a substantial gap between text-generation quality and recoverable procedural structure: recipes produced by existing approaches achieve competitive text-generation scores yet yield limited reference-aligned entity and relation structure under the ReGraph schema. In contrast, across two representative LMM backbones, RGL consistently improves the generation of cooking entities and procedural relations, while our analysis further shows that fine-grained ingredient-state capture remains the most challenging dimension.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。