arXiv:2608.19639cs.CV2026-08

用结构化稀疏高斯流提升边缘设备的自由视角视频重建效率

S$^2$GS: Structured Sparse Gaussian Streaming for Efficient Free-Viewpoint Video Reconstruction on Edge-IoT Devices

论文配图:S$^2$GS: Structured Sparse Gaussian Streaming for Efficient Free-Viewpoint Video Reconstruction on Edge-IoT Devices
图 1 · 摘自论文原文
  • 通过时空稀疏结构仅更新必要高斯残差,降低计算与存储开销
  • 在RTX 4090上将每帧优化时间减少59%,存储成本降低85%
  • 适合资源受限的物联网设备部署,如远程会议与数字孪生系统

自由视角视频(FVV)流式重建支持沉浸式物联网服务,如远程存在和数字孪生可视化。现有方法存在每帧优化时间长、存储开销大等问题,限制了在资源受限的边缘物联网设备上的部署。为此,我们提出结构化稀疏高斯流(S²GS)框架,利用结构感知的时间稀疏性,选择性地更新高斯残差,实现在不牺牲视觉保真度的前提下高效流式重建。空间域中,流式八叉树层级组织高斯残差,捕捉空间相关性以指导残差更新;时间域中,结构化门控机制结合层次特征传播(HFP)与Gumbel-Sigmoid采样,将层次动态信号转化为可微优化下的稀疏更新决策。进一步采用多级离散方案,实现对残差更新的细粒度控制,同时保留复杂动态细节。在消费级显卡、工业级边缘IoT设备及物理远程存在测试平台上的实验表明,S²GS持续降低每帧优化时间与存储开销,且保持优异视觉质量。相比QUEEN,在RTX 4090上每帧优化时间减少59%,存储成本降低85%;在Jetson AGX Orin上,渲染吞吐量超60 FPS,能耗最低,证明其在资源受限系统中的部署潜力。

原文摘要 · Abstract (English)

Streaming reconstruction of Free-Viewpoint Videos (FVVs) supports immersive Internet of Things (IoT) services, such as telepresence and digital twin visualization. Existing methods suffer from high per-frame optimization time and large storage footprints, limiting deployment on resource-constrained Edge-IoT devices. To address these challenges, we propose Structured Sparse Gaussian Streaming (S$^2$GS), an FVV reconstruction framework that exploits structure-aware temporal sparsity to selectively update Gaussian residuals, enabling efficient streaming without compromising visual fidelity. In the spatial domain, a streaming octree hierarchically organizes Gaussian residuals, capturing spatial correlations that guide residual updates. In the temporal domain, a structured gating mechanism, comprising hierarchical feature propagation (HFP) and Gumbel-Sigmoid sampling, converts hierarchical dynamic cues into sparse residual update decisions under differentiable optimization. A multi-level discrete scheme is further adopted to provide fine-grained control over residual updates while preserving intricate dynamic details. Extensive experiments across consumer GPUs, industrial edge IoT devices, and a physical telepresence testbed demonstrate that S$^2$GS consistently reduces per-frame optimization time and storage footprint while maintaining competitive visual quality. Compared with QUEEN, S$^2$GS reduces per-frame optimization time by 59% and storage costs by 85% on an RTX 4090 GPU. On the Jetson AGX Orin, S$^2$GS delivers the highest rendering throughput (60+ FPS) and the lowest energy consumption among the evaluated methods, demonstrating its potential for deployment in resource-constrained systems.

视频重建边缘计算稀疏优化IoT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。