用可学习修正项融合物理规律，让模型更准更可信地模拟真实动态。
Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics
- 在物理模型基础上加神经修正，兼顾准确性与可解释性。
- 在多种动态场景下，粒子定位精度显著提升，渲染质量也更好。
- 适合需要高可信度物理模拟的研究者或工业应用开发者。
人类能轻松识别内在动态并适应新场景，但当前AI系统常表现不佳。现有视觉动态建模方法要么采用纯神经网络模拟器（黑箱），可能违背物理规律；要么依赖专家定义方程的传统物理模拟器（白箱），却未必能完整捕捉真实动态。本文提出神经材料适配器（NeuMA），将已有物理规律与学习到的修正项结合，实现对实际动态的精准建模，同时保持物理先验的泛化性和可解释性。此外，我们提出粒子驱动的3D高斯溅射变体Particle-GS，连接模拟与观测图像，支持反向传播图像梯度以优化模拟器。在多种动态场景下的综合实验表明，NeuMA能准确捕捉内在动态，在粒子定位精度、动态渲染质量和泛化能力方面均表现优异。
原文摘要 · Abstract (English)
While humans effortlessly discern intrinsic dynamics and adapt to new scenarios, modern AI systems often struggle. Current methods for visual grounding of dynamics either use pure neural-network-based simulators (black box), which may violate physical laws, or traditional physical simulators (white box), which rely on expert-defined equations that may not fully capture actual dynamics. We propose the Neural Material Adaptor (NeuMA), which integrates existing physical laws with learned corrections, facilitating accurate learning of actual dynamics while maintaining the generalizability and interpretability of physical priors. Additionally, we propose Particle-GS, a particle-driven 3D Gaussian Splatting variant that bridges simulation and observed images, allowing back-propagate image gradients to optimize the simulator. Comprehensive experiments on various dynamics in terms of grounded particle accuracy, dynamic rendering quality, and generalization ability demonstrate that NeuMA can accurately capture intrinsic dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。