arXiv:2509.18165cs.CVcs.LG2025-09中稿 · Neural Networks 20…

通过逆映射重建输入,提升模型泛化能力。

Self Identity Mapping

  • 利用逆映射机制减少前向传播信息损失。
  • 在图像分类等任务上均显著优于基线方法。
  • 适合作为通用正则化模块嵌入各类模型与任务。

正则化对深度学习中的泛化能力和防过拟合至关重要,但传统方法多依赖启发式策略,在不同场景下效果不稳定。本文提出自身份映射(Self Identity Mapping, SIM),一种简单有效的数据内在正则化框架,通过从变换后的输出重构输入,减少前向传播中的信息丢失,促进更平滑的梯度流动。为解决计算效率问题,我们引入 $ρ\text{SIM}$,采用块级特征采样与基于投影的方法重构潜在特征,有效降低复杂度。作为模型无关、任务无关的正则化器,SIM可无缝作为即插即用模块集成于多种网络架构和任务中。我们在图像分类、少样本提示学习和域泛化三个任务上进行了广泛评估,实验结果表明 $ρ\text{SIM}$ 在各类任务中均一致优于基线方法,展现出强大的表示学习增强能力。此外,$ρ\text{SIM}$ 与现有正则化方法正交,能进一步提升其效果。结果还验证了其在密集到密集任务(如语义分割、图像翻译)及非视觉领域(如音频分类、时间序列异常检测)中有效保留语义信息并提升性能。代码已公开于 https://github.com/XiudingCai/SIM-pytorch。

原文摘要 · Abstract (English)

Regularization is essential in deep learning to enhance generalization and mitigate overfitting. However, conventional techniques often rely on heuristics, making them less reliable or effective across diverse settings. We propose Self Identity Mapping (SIM), a simple yet effective, data-intrinsic regularization framework that leverages an inverse mapping mechanism to enhance representation learning. By reconstructing the input from its transformed output, SIM reduces information loss during forward propagation and facilitates smoother gradient flow. To address computational inefficiencies, We instantiate SIM as $ ρ\text{SIM} $ by incorporating patch-level feature sampling and projection-based method to reconstruct latent features, effectively lowering complexity. As a model-agnostic, task-agnostic regularizer, SIM can be seamlessly integrated as a plug-and-play module, making it applicable to different network architectures and tasks. We extensively evaluate $ρ\text{SIM}$ across three tasks: image classification, few-shot prompt learning, and domain generalization. Experimental results show consistent improvements over baseline methods, highlighting $ρ\text{SIM}$'s ability to enhance representation learning across various tasks. We also demonstrate that $ρ\text{SIM}$ is orthogonal to existing regularization methods, boosting their effectiveness. Moreover, our results confirm that $ρ\text{SIM}$ effectively preserves semantic information and enhances performance in dense-to-dense tasks, such as semantic segmentation and image translation, as well as in non-visual domains including audio classification and time series anomaly detection. The code is publicly available at https://github.com/XiudingCai/SIM-pytorch.

正则化表示学习通用模块

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。