提出双速框架,让光谱压缩成像模型快速适应新设备,提升重建质量。
SlowFast-SCI: Slow-Fast Deep Unfolding Learning for Spectral Compressive Imaging
- 采用慢-快双阶段学习:慢速预训练提取通用先验,快速阶段自监督微调
- 在分布外数据上实现最高5.79 dB的PSNR提升,参数与计算量减少70%以上
- 适合部署于实际场景,尤其适用于未见过的光谱成像设备
人类学习包含缓慢积累知识与快速应对经验两种方式。现有光谱压缩成像(SCI)的深度展开方法仅模拟了慢速过程——依赖大量展开阶段的繁重预训练,却缺乏对新光学配置的快速适应能力,导致在分布外相机上性能下降,尤其在训练中未见的定制化光谱设置下表现不佳。这还带来高计算开销和慢推理速度。为此,我们提出SlowFast-SCI,一种可无缝嵌入任意深度展开网络的双速框架。慢速学习阶段,通过成像引导从基于先验的主干网络中蒸馏出紧凑的快速展开模型;快速学习阶段,在每个模块中嵌入轻量级适配模块,通过双域损失在测试时自监督训练,无需重训主干。据我们所知,SlowFast-SCI是首个面向高效、自适应光谱重建的测试时适配驱动型深度展开框架。其双阶段设计融合离线鲁棒性与实时样本级校准,实现参数与浮点运算量减少超70%,分布外数据上最多提升5.79 dB PSNR,保持跨域适应能力,且适配速度提升4倍。其模块化设计可集成至任意深度展开网络,为可自适应现场部署的成像系统及扩展计算成像模态铺路。代码与数据集详见https://github.com/XuanLu11/SlowFast-SCI。
原文摘要 · Abstract (English)
Humans learn in two complementary ways: a slow, cumulative process that builds broad, general knowledge, and a fast, on-the-fly process that captures specific experiences. Existing deep-unfolding methods for spectral compressive imaging (SCI) mirror only the slow component-relying on heavy pre-training with many unfolding stages-yet they lack the rapid adaptation needed to handle new optical configurations. As a result, they falter on out-of-distribution cameras, especially in bespoke spectral setups unseen during training. This depth also incurs heavy computation and slow inference. To bridge this gap, we introduce SlowFast-SCI, a dual-speed framework seamlessly integrated into any deep unfolding network beyond SCI systems. During slow learning, we pre-train or reuse a priors-based backbone and distill it via imaging guidance into a compact fast-unfolding model. In the fast learning stage, lightweight adaptation modules are embedded within each block and trained self-supervised at test time via a dual-domain loss-without retraining the backbone. To the best of our knowledge, SlowFast-SCI is the first test-time adaptation-driven deep unfolding framework for efficient, self-adaptive spectral reconstruction. Its dual-stage design unites offline robustness with on-the-fly per-sample calibration-yielding over 70% reduction in parameters and FLOPs, up to 5.79 dB PSNR improvement on out-of-distribution data, preserved cross-domain adaptability, and a 4x faster adaptation speed. In addition, its modularity integrates with any deep-unfolding network, paving the way for self-adaptive, field-deployable imaging and expanded computational imaging modalities. The models, datasets, and code are available at https://github.com/XuanLu11/SlowFast-SCI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。