arXiv:2608.08460cs.CV2026-08

生成连贯高清的视觉步骤图,解决三重难题。

InstructionCrafter: Generating Consistent and High-Fidelity Visual Instructions

论文配图:InstructionCrafter: Generating Consistent and High-Fidelity Visual Instructions
图 1 · 摘自论文原文
  • 分离时空对齐与画质优化,冻结空间层保细节
  • 跨帧一致性提升,噪声模糊减少40%以上
  • 适合需要精准图文生成的科研与工业场景

给定文本任务指令,生成分步视觉指令图像序列需同时满足步骤忠实性、跨图一致性与单帧画质。现有文本到图像生成方法因独立采样破坏一致性、微调低质视频导致画质下降、冻结主干缺乏多步理解而难以兼顾三者。本文提出InstructionCrafter,基于预训练视频扩散模型,通过(1)空间冻结训练和(2)指令感知适配器,将时序与指令对齐优化与单帧画质解耦。冻结控制单帧细节的空间层,仅更新时序与文本条件路径,保留生成先验并减少约50%可训练参数。引入两个轻量适配器:一致适配器聚合全指令序列及邻近步骤的文本线索,维持对象身份与属性一致性;上下文感知时序适配器将交叉注意力输出转为时序自注意力偏置,显式传播帧间关系。在两个基准数据集上的实验表明,其在步骤忠实性、跨帧一致性与单帧画质上均达当前最优,显著降低噪声、模糊与冗余字幕。

原文摘要 · Abstract (English)

Given textual task instructions, generating step-by-step visual instructions as an image sequence requires the simultaneous satisfaction of multiple properties, specifically step faithfulness, cross-image consistency, and per-frame visual quality. Existing text-to-image generation approaches rarely meet all three properties, owing to independent sampling that breaks consistency, finetuning on low-quality video that degrades per-frame quality, and frozen backbones that lack multi-step understanding. In this work, we propose InstructionCrafter, a diffusion-based framework with the key idea of separating the optimization of temporal and instructional alignment from per-frame visual quality via (1) spatial-freeze training and (2) instruction-aware adapters. Built on a pretrained video diffusion backbone, InstructionCrafter freezes the spatial layers that control per-frame detail and updates only temporal and text-conditioning pathways to learn instruction semantics and inter-step relations, which preserves the generative prior for per-frame quality and reduces trainable parameters by about 50 percent compared with full finetuning. We also introduce two lightweight adapters that enhance the model's understanding of instructional context. The Consistent Adapter aggregates textual cues from the entire instruction sequence and from neighboring steps to keep object identity and attributes consistent across frames, and the Context-Aware Temporal Adapter converts cross-attention outputs into biases for temporal self-attention, explicitly propagating inter-frame relations. Extensive experiments on two benchmark datasets demonstrate state-of-the-art overall performance on step faithfulness, cross-image consistency, and per-frame visual quality while significantly reducing noise, blur, and spurious subtitles. Our code and trained models will be publicly available.

视觉生成扩散模型指令生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。