用局部顺序损失提升自编码器重建精度,让压缩结果更准。
Local-Order Auxiliary Losses Can Improve Autoencoder Reconstruction
- 引入符号差分损失(FDSE),惩罚邻近点差分符号不一致
- 混合MSE与FDSE后验证误差降低2.3至7.0倍
- 适合需要保持空间连贯性的信号重建任务
均方误差是自编码器训练的默认目标,但压缩重建不仅依赖逐点精度,还需保持局部空间顺序。我们研究结构化辅助损失能否在有限容量自编码器中提升而非替代MSE。提出有限差分符号误差(FDSE),通过惩罚目标与重构中邻近差分符号的不一致来优化局部结构。该方法简单、与架构无关,且可通过平滑符号代理实现可微。在四个张量重建任务中,适度混合MSE与FDSE可显著降低验证集MSE。系数扫描显示,相较纯MSE,FDSE混合方案在这些任务上使验证MSE降低2.3×至7.0×;与其他辅助目标对比,FDSE为最强结构化目标之一。该效果并非普遍:纯FDSE表现差,增益在具有连贯空间场的任务中最大。结果表明,在压缩潜在表示重建中,合理加权的局部结构监督可引导优化获得更高逐点精度,而非仅提升感知或结构性指标而牺牲MSE性能。
原文摘要 · Abstract (English)
Mean-squared error is the default objective for training autoencoders, yet compressed reconstructions often depend not only on pointwise accuracy but also on preserving local spatial order. We study whether structural auxiliary losses can improve, rather than trade off against, MSE in finite-capacity autoencoders. We introduce finite-difference sign error (FDSE), a local-order auxiliary objective that penalizes disagreements between the signs of neighboring finite differences in the target and reconstruction. FDSE is simple, architecture-agnostic, and differentiable through smooth sign surrogates. Across four tensor reconstruction tasks, we find that moderate mixtures of MSE and FDSE can substantially reduce validation MSE relative to pure MSE training. In coefficient sweeps, FDSE mixtures reduce validation MSE by 2.3$\times$--7.0$\times$ over pure MSE on these tasks, while comparisons with other auxiliary objectives show FDSE to be among the strongest structural objectives tested. The effect is not universal: pure FDSE performs poorly, and gains are largest for coherent spatial fields where local order carries information about the underlying signal. These results suggest that, in compressed-latent reconstruction, appropriately weighted local-structure supervision can guide optimization toward solutions with better pointwise accuracy, rather than merely improving perceptual or structural metrics at MSE's expense.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。