用预测初值加速稀疏编码,让算法更快更准。
WARP-LCA: Efficient Convolutional Sparse Coding with Locally Competitive Algorithm
- 用预测网络提供初始状态,替代传统盲目迭代。
- 收敛速度提升数量级,解的质量更高且更稀疏。
- 适合图像去噪与深度识别任务,生物启发模型优化新范式。
局部竞争算法(LCA)可广泛应用于稀疏编码问题。近期基于卷积的LCA方法在视觉流水线中显著提升了图像识别的鲁棒性。为进一步增强表示稀疏性,可结合硬阈值化,但该组合存在明显缺陷:(i) LCA效率极低,通常需数百次优化循环才能收敛;(ii) 硬阈值导致非凸损失函数,易陷入次优极小值。为此,我们提出带状态预热的预测引导局部竞争算法(WARP-LCA),利用预测网络根据当前输入生成合适的LCA初始状态。该方法大幅提高收敛速度与解的质量,同时保持并增强LCA原有优势。实验表明,WARP-LCA收敛速度比传统LCA快多个数量级,达到更优极小值。所学表示更稀疏,在重建、去噪及深度识别流水线中表现出更优性能。此外,我们将WARP-LCA应用于图像去噪任务,验证其鲁棒性与实用性。结果证实,直接使用硬阈值化LCA会陷入次优解,而通过预测初值初始化则能获得更好结果。本研究推动了生物启发深度学习在卷积稀疏编码中的发展。
原文摘要 · Abstract (English)
The locally competitive algorithm (LCA) can solve sparse coding problems across a wide range of use cases. Recently, convolution-based LCA approaches have been shown to be highly effective for enhancing robustness for image recognition tasks in vision pipelines. To additionally maximize representational sparsity, LCA with hard-thresholding can be applied. While this combination often yields very good solutions satisfying an $\ell_0$ sparsity criterion, it comes with significant drawbacks for practical application: (i) LCA is very inefficient, typically requiring hundreds of optimization cycles for convergence; (ii) the use of hard-thresholding results in a non-convex loss function, which might lead to suboptimal minima. To address these issues, we propose the Locally Competitive Algorithm with State Warm-up via Predictive Priming (WARP-LCA), which leverages a predictor network to provide a suitable initial guess of the LCA state based on the current input. Our approach significantly improves both convergence speed and the quality of solutions, while maintaining and even enhancing the overall strengths of LCA. We demonstrate that WARP-LCA converges faster by orders of magnitude and reaches better minima compared to conventional LCA. Moreover, the learned representations are more sparse and exhibit superior properties in terms of reconstruction and denoising quality as well as robustness when applied in deep recognition pipelines. Furthermore, we apply WARP-LCA to image denoising tasks, showcasing its robustness and practical effectiveness. Our findings confirm that the naive use of LCA with hard-thresholding results in suboptimal minima, whereas initializing LCA with a predictive guess results in better outcomes. This research advances the field of biologically inspired deep learning by providing a novel approach to convolutional sparse coding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。