用单步回归+单步扩散重建一比特调制的视频快照成像,解决时间混叠问题。
3One2: One-step Regression Plus One-step Diffusion for One-hot Modulation in Dual-path Video Snapshot Compressive Imaging
- 将一比特调制重构转为生成式补全,设计匹配硬件的随机微分方程。
- 结合单步回归初始化与单步扩散优化,提升重建质量并减少空间退化。
- 首创扩散模型用于视频快照成像,适合高动态场景重建任务。
视频快照压缩成像(SCI)通过二维快照捕捉动态场景序列,依赖光学调制实现硬件压缩并辅以软件重建。主流随机二值调制虽有效,但不可避免引发时间混叠。一比特调制(仅每像素激活一个子帧)可实现理想的时间解耦,缓解混叠问题,但当前尚无算法充分挖掘其潜力。为此,我们提出专为一比特掩码设计的算法:首先,利用一比特调制的解耦特性,将重建任务转化为生成式视频补全问题,并构建与硬件压缩过程一致的前向过程随机微分方程(SDE)。其次,发现纯扩散方法在视频SCI中存在局限,提出融合单步回归初始化与单步扩散精修的新框架。此外,为缓解一比特调制导致的空间退化,硬件层面采用双光路设计,利用另一路径的互补信息增强补全效果。据我们所知,这是首个将扩散模型引入视频快照成像重建的工作。在合成数据集和真实场景上的实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Video snapshot compressive imaging (SCI) captures dynamic scene sequences through a two-dimensional (2D) snapshot, fundamentally relying on optical modulation for hardware compression and the corresponding software reconstruction. While mainstream video SCI using random binary modulation has demonstrated success, it inevitably results in temporal aliasing during compression. One-hot modulation, activating only one sub-frame per pixel, provides a promising solution for achieving perfect temporal decoupling, thereby alleviating issues associated with aliasing. However, no algorithms currently exist to fully exploit this potential. To bridge this gap, we propose an algorithm specifically designed for one-hot masks. First, leveraging the decoupling properties of one-hot modulation, we transform the reconstruction task into a generative video inpainting problem and introduce a stochastic differential equation (SDE) of the forward process that aligns with the hardware compression process. Next, we identify limitations of the pure diffusion method for video SCI and propose a novel framework that combines one-step regression initialization with one-step diffusion refinement. Furthermore, to mitigate the spatial degradation caused by one-hot modulation, we implement a dual optical path at the hardware level, utilizing complementary information from another path to enhance the inpainted video. To our knowledge, this is the first work integrating diffusion into video SCI reconstruction. Experiments conducted on synthetic datasets and real scenes demonstrate the effectiveness of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。