arXiv:2511.11031cs.CVcs.MM2025-11被引 1

通过混合粒度缓存加速可控生成,显著降低计算开销。

Accelerating Controllable Generation via Hybrid-grained Cache

  • 采用块级与提示级双重缓存策略,动态跳过冗余计算。
  • 在COCO-Stuff上减少63%计算量(18.22T→6.70T),质量损失<1.5%。
  • 适合需要高效可控生成的视觉合成场景,如图像编辑与视频生成。

可控生成模型广泛用于提升合成视觉内容的真实感,但需同时处理控制条件与生成计算,导致效率低下。为此,我们提出混合粒度缓存(HGC)方法,通过在不同计算阶段采用不同粒度的缓存策略降低计算开销。具体地,(1) 基于特征复用,使用块级粗粒度缓存,在编码器-解码器块间推理步骤中动态跳过冗余计算;(2) 设计提示级细粒度缓存,于模块内复用连续推理步骤中的交叉注意力图,并延伸至相邻步骤的模块计算。不同粒度的缓存可无缝集成到可控生成流程的每个计算链路中。我们在四个基准数据集上验证了HGC的有效性,尤其在生成效率与视觉质量间的平衡表现优异。例如,在COCO-Stuff分割任务上,计算量(MACs)降低63%(从18.22T降至6.70T),语义保真度损失(量化性能退化)控制在1.5%以内。

原文摘要 · Abstract (English)

Controllable generative models have been widely used to improve the realism of synthetic visual content. However, such models must handle control conditions and content generation computational requirements, resulting in generally low generation efficiency. To address this issue, we propose a Hybrid-Grained Cache (HGC) approach that reduces computational overhead by adopting cache strategies with different granularities at different computational stages. Specifically, (1) we use a coarse-grained cache (block-level) based on feature reuse to dynamically bypass redundant computations in encoder-decoder blocks between each step of model reasoning. (2) We design a fine-grained cache (prompt-level) that acts within a module, where the fine-grained cache reuses cross-attention maps within consecutive reasoning steps and extends them to the corresponding module computations of adjacent steps. These caches of different granularities can be seamlessly integrated into each computational link of the controllable generation process. We verify the effectiveness of HGC on four benchmark datasets, especially its advantages in balancing generation efficiency and visual quality. For example, on the COCO-Stuff segmentation benchmark, our HGC significantly reduces the computational cost (MACs) by 63% (from 18.22T to 6.70T), while keeping the loss of semantic fidelity (quantized performance degradation) within 1.5%.

生成模型缓存优化视觉合成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。