轻量级Transformer框架实现高效无伪影的HDR重建。
EfficienT-HDR: An Efficient Transformer-Based Framework via Multi-Exposure Fusion for HDR Reconstruction
- 基于多曝光融合与自适应注意力机制,分离亮度和色彩信息。
- 主版本减少67%计算量,推理速度提升五倍以上。
- 适合边缘设备部署,适用于智能监控与自动驾驶场景。
在资源受限的边缘设备上实现高质量高动态范围(HDR)成像仍是计算机视觉的关键挑战,其性能直接影响智能监控、自动驾驶等下游任务。多曝光融合(MEF)是主流方法,但现有技术普遍存在计算开销大和鬼影伪影问题,限制了实际应用。本文提出一种专为HDR重建设计的轻量级视觉变换器架构。基于上下文感知视觉变换器,将输入图像转换至YCbCr色彩空间以分离亮度与色度信息,并引入交集感知自适应融合(IAAF)模块有效抑制鬼影。为降低复杂度,提出逆残差嵌入(IRE)、动态Tanh(DyT)和增强多尺度空洞卷积(E-MSDC),在多个层级优化计算效率。最终构建两个版本:主模型追求高视觉质量,轻量版侧重计算效率,均实现性能与质量的良好平衡。实验表明,相比基线,主模型在CPU上减少约67%浮点运算量(FLOPS),推理速度提升逾五倍;在边缘设备上提速2.5倍。结果验证了该方法在各类动态场景中具备高效、无伪影的实用能力。
原文摘要 · Abstract (English)
Achieving high-quality High Dynamic Range (HDR) imaging on resource-constrained edge devices is a critical challenge in computer vision, as its performance directly impacts downstream tasks such as intelligent surveillance and autonomous driving. Multi-Exposure Fusion (MEF) is a mainstream technique to achieve this goal; however, existing methods generally face the dual bottlenecks of high computational costs and ghosting artifacts, hindering their widespread deployment. To this end, this study proposes a light-weight Vision Transformer architecture designed explicitly for HDR reconstruction to overcome these limitations. This study is based on the Context-Aware Vision Transformer and begins by converting input images to the YCbCr color space to separate luminance and chrominance information. It then employs an Intersection-Aware Adaptive Fusion (IAAF) module to suppress ghosting effectively. To further achieve a light-weight design, we introduce Inverted Residual Embedding (IRE), Dynamic Tanh (DyT), and propose Enhanced Multi-Scale Dilated Convolution (E-MSDC) to reduce computational complexity at multiple levels. Our study ultimately contributes two model versions: a main version for high visual quality and a light-weight version with advantages in computational efficiency, both of which achieve an excellent balance between performance and image quality. Experimental results demonstrate that, compared to the baseline, the main version reduces FLOPS by approximately 67% and increases inference speed by more than fivefold on CPU and 2.5 times on an edge device. These results confirm that our method provides an efficient and ghost-free HDR imaging solution for edge devices, demonstrating versatility and practicality across various dynamic scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。