用光照与语义先验增强低光图像，提升真实场景下的细节还原。
ISALux: Illumination and Segmentation Aware Transformer Employing Mixture of Experts for Low Light Image Enhancement
- 融合光照和语义图的自注意力模块，分途处理并相互调节。
- 引入专家混合机制，按需激活专业模块提升上下文理解能力。
- 适合需要高保真细节恢复的低光图像应用，如夜视与医疗成像。
我们提出ISALux，一种基于Transformer的低光图像增强新方法，深度融合光照与语义先验。其核心为新型自注意力模块HISA-MSA，通过联合处理光照图与语义分割图，实现更优特征提取。模型采用双自注意力分支独立建模光照与语义信息，并动态交互以调节亮度与结构变化。引入基于门控机制的专家混合（MoE）前馈网络，仅激活前K个专家，实现高效专业化处理。为缓解基准数据集光照模式差异导致的过拟合问题，对HISA-MSA模块引入低秩矩阵适配（LoRA）。在多个专用数据集上的定量与定性评估表明，ISALux性能媲美当前最优方法。消融实验验证了各组件的有效性。代码将在论文发表后公开。
原文摘要 · Abstract (English)
We introduce ISALux, a novel transformer-based approach for Low-Light Image Enhancement (LLIE) that seamlessly integrates illumination and semantic priors. Our architecture includes an original self-attention block, Hybrid Illumination and Semantics-Aware Multi-Headed Self- Attention (HISA-MSA), which integrates illumination and semantic segmentation maps for en- hanced feature extraction. ISALux employs two self-attention modules to independently process illumination and semantic features, selectively enriching each other to regulate luminance and high- light structural variations in real-world scenarios. A Mixture of Experts (MoE)-based Feed-Forward Network (FFN) enhances contextual learning, with a gating mechanism conditionally activating the top K experts for specialized processing. To address overfitting in LLIE methods caused by distinct light patterns in benchmarking datasets, we enhance the HISA-MSA module with low-rank matrix adaptations (LoRA). Extensive qualitative and quantitative evaluations across multiple specialized datasets demonstrate that ISALux is competitive with state-of-the-art (SOTA) methods. Addition- ally, an ablation study highlights the contribution of each component in the proposed model. Code will be released upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。