让神经网络预测的动态可被简洁符号公式描述,提升可解释性与稳定性。
SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictors

- 用符号规律+正则化神经修正混合建模动态,兼顾简洁与精度。
- 在摆动实验中,联合学习比事后拟合更简单、误差更低、发散更少。
- 适合追求模型可解释性与可控动态的科研与工程应用。
联合嵌入预测架构通过从上下文嵌入预测目标嵌入来学习抽象状态,但其转换模型通常为不可解释的神经映射。我们提出SJEPA,一种无需重建的JEPA框架,所学预测表示诱导出可紧凑符号描述的动力学。其混合转换结合了符号规律与正则化神经修正,用于处理选定语法外的动力学。核心原则是学习最简且足够的动力学:表示约束保留有信息量、不坍缩的预测坐标,而算子压缩偏好低复杂度的符号-神经转换,同时保持预测有效性。我们通过诱导动力学复杂度形式化该原则,分析预测坐标的非辨识性,并表明无约束的算子压缩会直接导致表示坍缩。框架支持交替表示-方程学习及固定表示下的符号动力学拟合。在受控摆实验中,联合学习发现显著更简单的符号动力学,长期滚动预测误差和发散均更低;而无约束的一步诊断验证了预测的坍缩捷径。当语法误设时,修正正则化能保留可表示的符号机制,并引导神经部分关注残差动力学。结果揭示了预测保真度、表示质量、符号简约性与符号-神经分配之间的可控权衡。
原文摘要 · Abstract (English)
Joint-embedding predictive architectures learn abstract states by predicting target embeddings from context embeddings, but their transition models are typically opaque neural maps. We introduce SJEPA, a reconstruction-free JEPA framework that learns predictive representations whose induced dynamics admit compact symbolic descriptions. Its hybrid transition combines a symbolic law with a regularised neural correction for dynamics outside the selected grammar. The central principle is to learn the simplest adequate dynamics: representation constraints preserve informative, non-collapsed predictive coordinates, while operator compression favours low-complexity symbolic-neural transitions that remain predictively adequate. We formalise this principle through induced-dynamics complexity, analyse predictive-coordinate non-identifiability, and show that unconstrained operator compression creates a direct shortcut to representation collapse. The framework supports both alternating representation-equation learning and symbolic dynamics fitted to fixed representations. In controlled pendulum experiments, joint learning discovers substantially simpler symbolic dynamics with lower long-horizon rollout error and divergence than post-hoc fitting, while an unconstrained one-step diagnostic realises the predicted collapse shortcut. Under grammar misspecification, correction regularisation preserves the representable symbolic mechanism and directs the neural component towards residual dynamics. The results expose a controllable trade-off among predictive fidelity, representation quality, symbolic parsimony, and symbolic-neural allocation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。