提出新型门控预测自编码器，在噪声环境下表现优于JEPA。
Is the reconstruction loss culprit? An attempt to outperform JEPA
- 设计门控机制，自动选择可预测的特征分量。
- 在不同噪声水平下稳定表现，性能超越或媲美JEPA。
- 为预测学习提供新思路，适合关注模型鲁棒性的研究者。
我们在一个具有已知潜在状态和单一噪声参数的受控线性动态系统（TV-series）上，评估了基于预测的表示学习方法（如JEPA）与基于重构的自编码器。初步结果显示JEPA对噪声更鲁棒，但深入分析发现自编码器失败主要源于目标函数不对称性及瓶颈层/组件选择效应（通过PCA基线验证）。基于此，我们提出门控预测自编码器，能够学习选择可预测的组件，模仿过参数化PCA中的有益特征选择行为。在该测试平台中，所提模型在不同噪声水平下均保持稳定，性能达到或超过JEPA。
原文摘要 · Abstract (English)
We evaluate JEPA-style predictive representation learning versus reconstruction-based autoencoders on a controlled "TV-series" linear dynamical system with known latent state and a single noise parameter. While an initial comparison suggests JEPA is markedly more robust to noise, further diagnostics show that autoencoder failures are strongly influenced by asymmetries in objectives and by bottleneck/component-selection effects (confirmed by PCA baselines). Motivated by these findings, we introduce gated predictive autoencoders that learn to select predictable components, mimicking the beneficial feature-selection behavior observed in over-parameterized PCA. On this toy testbed, the proposed gated model is stable across noise levels and matches or outperforms JEPA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。