让视觉生成模型记住前块状态,显著提升图像质量。
Arcee: Differentiable Recurrent State Chain for Generative Vision Modeling with Mamba SSMs
- 跨块传递终态信息,构建可微分的递归状态链
- 在CelebA-HQ上将FID从82.81降至15.33(降5.4倍)
- 无需额外参数,适配所有视觉Mamba模型
状态空间模型(SSM),尤其是Mamba,在长序列建模中日益流行,通过输入相关的因果选择性扫描实现线性时间聚合。目前的“Mamba-视觉”变体多尝试多种扫描顺序以放松对非序列信号(如图像)的严格因果性。然而,传统Mamba的选择性扫描在每块间重置状态空间动态,丢弃前一块的终态表示(SSR)。Arcee提出一种跨块递归状态链,将每块的终态表示作为下一块的初始条件。块间传递通过可微分边界映射实现,其雅可比矩阵支持端到端梯度流。关键优势在于,Arcee兼容所有既有“视觉Mamba”方法,无额外参数,开销恒定且可忽略。从建模视角看,终态SSR被视为因果遍历输入带来的温和方向先验,而非非序列信号的估计。量化验证显示,在256×256的CelebA-HQ无条件生成任务中,基于流匹配(Flow Matching),Arcee将FID从82.81降至15.33(降低5.4倍),性能显著提升。高效CUDA内核与训练代码将公开,支持严谨可复现研究。
原文摘要 · Abstract (English)
State-space models (SSMs), Mamba in particular, are increasingly adopted for long-context sequence modeling, providing linear-time aggregation via an input-dependent, causal selective-scan operation. Along this line, recent "Mamba-for-vision" variants largely explore multiple scan orders to relax strict causality for non-sequential signals (e.g., images). Rather than preserving cross-block memory, the conventional formulation of the selective-scan operation in Mamba reinitializes each block's state-space dynamics from zero, discarding the terminal state-space representation (SSR) from the previous block. Arcee, a cross-block recurrent state chain, reuses each block's terminal state-space representation as the initial condition for the next block. Handoff across blocks is constructed as a differentiable boundary map whose Jacobian enables end-to-end gradient flow across terminal boundaries. Key to practicality, Arcee is compatible with all prior "vision-mamba" variants, parameter-free, and incurs constant, negligible cost. As a modeling perspective, we view terminal SSR as a mild directional prior induced by a causal pass over the input, rather than an estimator of the non-sequential signal itself. To quantify the impact, for unconditional generation on CelebA-HQ (256$\times$256) with Flow Matching, Arcee reduces FID$\downarrow$ from $82.81$ to $15.33$ ($5.4\times$ lower) on a single scan-order Zigzag Mamba baseline. Efficient CUDA kernels and training code will be released to support rigorous and reproducible research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。