用物理启发的多模态模型预测无界表面生长，实现对世界动态的智能理解。
FOLIAGE: Towards Physical Intelligence World Models Via Unbounded Surface Evolution
- 统一编码器融合图像、网格和点云，生成共享隐状态。
- 在7200个生长序列上表现优于专用基线，跨任务保持鲁棒性。
- 适合研究物理世界模型、多模态生成与具身智能的开发者。
物理智能——从部分多感官观测中预测并塑造世界——是下一代世界模型的关键。我们提出FOLIAGE，一种用于无界累积表面生长的物理信息多模态世界模型。其动作-感知循环中，统一上下文编码器将图像、网格连接性和点云映射到共享隐状态。基于物理控制动作的物理感知预测器推进该隐状态以匹配目标隐状态，生成与模态无关的生长嵌入（MAGE），并与判别头对接下游目标。FOLIAGE的累积图网络（AGN）通过年龄位置编码与能量门控消息传递捕捉动态连接性。几何对应融合与跨块掩码增强MAGE表达力，层级池化平衡全局上下文与局部动态。我们构建了SURF-GARDEN平台，包含反事实物理模拟器、多模态对应提取器和演化追踪模块，生成7200个多样化的表面生长序列。SURF-BENCH评估套件涵盖六项核心任务：拓扑识别、逆材料估计、生长阶段分类、隐状态滚动、跨模态检索与密集对应，以及四项压力测试：传感器丢失、零样本模态迁移、长时序预测与物理消融，以检验模型韧性。FOLIAGE在多个任务中超越专用基线，同时在动态环境中保持鲁棒性，确立了一条基于世界模型的多模态物理智能新路径。
原文摘要 · Abstract (English)
Physical intelligence -- anticipating and shaping the world from partial, multisensory observations -- is critical for next-generation world models. We propose FOLIAGE, a physics-informed multimodal world model for unbounded accretive surface growth. In its Action-Perception loop, a unified context encoder maps images, mesh connectivity, and point clouds to a shared latent state. A physics-aware predictor, conditioned on physical control actions, advances this latent state in time to align with the target latent of the surface, yielding a Modality-Agnostic Growth Embedding (MAGE) that interfaces with critic heads for downstream objectives. FOLIAGE's Accretive Graph Network (AGN) captures dynamic connectivity through Age Positional Encoding and Energy-Gated Message-Passing. Geometry-Correspondence Fusion and Cross-Patch Masking enhance MAGE's expressiveness, while Hierarchical Pooling balances global context with local dynamics. We create SURF-GARDEN, a world model learning platform comprising a Counterfactual Physics Simulator, a Multimodal Correspondence Extractor, and Evolution Tracing, which generates 7,200 diverse surface-growth sequences. SURF-BENCH, our physical-intelligence evaluation suite, evaluates six core tasks -- topology recognition, inverse material estimation, growth-stage classification, latent roll-out, cross-modal retrieval, and dense correspondence -- and four stress tests -- sensor dropout, zero-shot modality transfer, long-horizon prediction, and physics ablation -- to probe resilience. FOLIAGE outperforms specialized baselines while remaining robust across dynamic environments, establishing a new world-model based, multimodal pathway to physical intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。