提出视频级光谱重建新方法，提升动态场景下的精度与时间一致性。
Exploring Spatiotemporal Feature Propagation for Video-Level Compressive Spectral Reconstruction: Dataset, Model and Benchmark
- 采用时空分步注意力机制，利用视频帧间连续性增强重建
- 在真实数据集上实现更高光谱保真度和时间一致性，仅用少量计算量
- 构建首个高质量动态高光谱数据集DynaSpec，支持真实世界测试
近年来，光谱压缩成像（SCI）取得显著进展，为动态光谱视觉开辟了新可能。然而，现有重建方法多基于图像级处理，存在两大缺陷：(i) 编码过程掩盖空间-光谱特征，导致单次压缩测量下缺失信息难以恢复；(ii) 帧独立重建无法保证时间一致性，影响视频感知效果。为此，本文将光谱重建从图像层面推进至视频层面，利用动态场景中相邻帧间的互补特征与时间连续性。首先，构建首个高质量动态高光谱图像数据集DynaSpec，包含30个序列，通过帧扫描采集获得。其次，提出传播引导的光谱视频重建变换器PG-SVRT，采用空间-然后-时间注意力机制，有效从丰富视频信息中恢复光谱特征，并引入桥接标记降低计算复杂度。最后，通过模拟实验评估四种SCI系统性能，并搭建真实世界原型DD-CASSI进行数据采集与基准测试。大量实验证明，PG-SVRT在重建质量、光谱保真度与时间一致性方面表现更优，同时保持极低浮点运算量（FLOPs）。项目主页：https://github.com/nju-cite/DynaSpec
原文摘要 · Abstract (English)
Recently, Spectral Compressive Imaging (SCI) has achieved remarkable success, unlocking significant potential for dynamic spectral vision. However, existing reconstruction methods, primarily image-based, suffer from two limitations: (i) Encoding process masks spatial-spectral features, leading to uncertainty in reconstructing missing information from single compressed measurements, and (ii) The frame-by-frame reconstruction paradigm fails to ensure temporal consistency, which is crucial in the video perception. To address these challenges, this paper seeks to advance spectral reconstruction from the image level to the video level, leveraging the complementary features and temporal continuity across adjacent frames in dynamic scenes. Initially, we construct the first high-quality dynamic hyperspectral image dataset (DynaSpec), comprising 30 sequences obtained through frame-scanning acquisition. Subsequently, we propose the Propagation-Guided Spectral Video Reconstruction Transformer (PG-SVRT), which employs a spatial-then-temporal attention to effectively reconstruct spectral features from abundant video information, while using a bridged token to reduce computational complexity. Finally, we conduct simulation experiments to assess the performance of four SCI systems, and construct a DD-CASSI prototype for real-world data collection and benchmarking. Extensive experiments demonstrate that PG-SVRT achieves superior performance in reconstruction quality, spectral fidelity, and temporal consistency, while maintaining minimal FLOPs. Project page: https://github.com/nju-cite/DynaSpec
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。