提出双通道世界模型,解决多任务学习中信号干扰导致的表征坍塌问题。
Dual-Channel Grounded World Modeling (DCGWM): Structural Prevention of Objective Interference Collapse via Heterogeneous External Grounding with Inward-Only Gradient Flow
- 用分离的物理与行为子空间,仅允许单向梯度流防止干扰
- 物理通道用约束对齐,行为通道用多智能体模拟对齐,互不传递梯度
- 适用于需要同时建模物理规律与社会行为的复杂场景
联合嵌入预测架构(JEPAs)是世界模型表征学习的主流方法。我们识别出基于JEPA的世界模型在对接两种性质迥异的外部信号时存在失效模式:物理动力学(稀疏、高幅值、满足约束的梯度修正)和社会行为动力学(弥散、分布匹配的修正)。我们称之为目标干扰坍塌(OIC):共享潜空间中的联合学习会使主导通道系统性压缩从属通道的表征子空间,仅靠损失权重无法解决。为此提出双通道接地世界建模(DCGWM),通过分区潜空间(物理子空间Z_p,行为子空间Z_b)和仅向内梯度流,结构化预防OIC。物理接地通道仅通过VICReg式对齐更新Z_p;社会行为接地通道仅通过涌现多智能体模拟轨迹对齐更新Z_b。跨通道接口模块在任务层面耦合子空间,但无跨子空间梯度。非对称接地遵从损失对物理违规使用硬铰链惩罚,对行为偏离使用软KL惩罚。生成渲染层在架构上与潜世界模型隔离。本文给出三项理论结果:分区移除了导致OIC的梯度干扰路径;每个接地子空间继承其对齐目标的抗坍塌保证;在生成目标几何的给定假设下,生成隔离是必要的。本稿建立问题形式化与架构;实验验证正在进行,将在后续版本报告。
原文摘要 · Abstract (English)
Joint Embedding Predictive Architectures (JEPAs) are a leading approach to world model representation learning. We identify a failure mode in JEPA-based world models grounded against two qualitatively distinct external signals: physical dynamics (sparse, high-magnitude, constraint-satisfying gradient corrections) and social-behavioral dynamics (diffuse, distribution-matching corrections). We term this Objective Interference Collapse (OIC): we argue that joint learning in a shared latent space causes the dominant channel to systematically collapse the subordinate channel's representational subspace, in a manner not resolvable by loss weighting alone. We propose Dual-Channel Grounded World Modeling (DCGWM), designed to structurally prevent OIC through a partitioned latent space (physical subspace Z_p, behavioral subspace Z_b) with inward-only gradient flow. A Physical Grounding Channel updates only Z_p via VICReg-style alignment to physical measurements; a Social-Behavioral Grounding Channel updates only Z_b via alignment to trajectories from an emergent multi-agent simulation. An Inter-Channel Interface Module couples the subspaces at the task level without cross-subspace gradients. An Asymmetric Grounding Adherence Loss penalizes rollout drift with a hard hinge for physical violations and a soft KL for behavioral divergence. A Generative Rendering Layer is architecturally isolated from the latent world model. We present three theoretical results: the partition removes the gradient-interference pathway implicated in OIC; each grounded subspace inherits anti-collapse guarantees from its alignment objective; and generative isolation is necessary under a stated assumption on the generative objective's geometry. This manuscript establishes the problem formulation and architecture; experimental validation is ongoing and will be reported in a future revision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。