arXiv:2602.07064cs.CV2026-02被引 1

让AI通过多模态信号理解物理规律,实现自主进化。

OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization

  • 构建多模态统一网络,融合图像、音频、视频与文本感知。
  • 通过物理规则约束生成高质量训练数据,提升模型物理理解能力。
  • 适合研究多模态学习与具身智能的学者参考。

网络化AI系统的自主演化依赖于稳健的环境感知,但现有模型对物理世界的理解仍不稳固,因关键物理信号在视觉上模糊且在大规模网络数据中稀疏分布。为此,我们提出OmniFysics——一个紧凑的全模态网络,统一处理图像、音频、视频与文本信号。为实现自主优化并注入显式物理知识,我们构建动态物理数据引擎:其中,FysicsAny通过分层检索与物理定律约束的信号验证,将显著物体映射至可验证的物理属性,生成基于物理的监督信号;FysicsOmniCap则利用先进的音视频跨模态信号处理,从网络视频中提炼出强调动态物理线索的高保真数据对。通过分阶段多模态对齐与演进式指令微调优化OmniFysics网络,结合潜在空间流匹配生成机制与自适应意图路由,实现高效执行。实验表明,该演进式优化范式不仅在标准多模态基准上表现优异,更显著提升物理导向评估性能。

原文摘要 · Abstract (English)

The autonomous evolution of networked AI systems relies heavily on robust environmental perception. However, physical understanding remains brittle in current models because key physical signals are visually ambiguous and sparsely represented in web-scale data. To bridge the gap between data-centric learning and knowledge-based physical rules, we present OmniFysics, a compact omni-modal network that unifies signal processing and understanding across images, audio, video, and text. To enable autonomous optimization and inject explicit physical knowledge, we construct a dynamic physical data engine. Within this engine, FysicsAny acts as an adaptive mechanism that produces physics-grounded supervision by mapping salient objects to verified physical attributes via hierarchical retrieval and physics-law-constrained signal verification. Concurrently, FysicsOmniCap distills web videos utilizing advanced audio-visual cross-modal signal processing, generating high-fidelity data pairs that emphasize dynamic physical cues. We optimize the OmniFysics network through staged multimodal alignment and evolutive instruction tuning, integrating latent-space flow matching for generation and an adaptive intent router for efficient execution. Experiments demonstrate that this evolutive optimization paradigm not only achieves competitive performance on standard multimodal benchmarks but also significantly advances physics-oriented evaluations.

多模态物理理解自进化生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。