OmniField让模型在缺失或噪声传感器数据下仍能准确预测多模态时空数据。
OmniField: Conditioned Neural Fields for Robust Multimodal Spatiotemporal Learning
- 用条件神经场和迭代跨模态融合，自动适应不同时间空间的传感器组合。
- 在强模拟噪声下性能仍接近干净数据，显著优于8个基线模型。
- 适合处理真实世界中传感器不完整、有噪声的多模态时空数据场景。
真实世界实验数据的多模态时空学习面临两大挑战：同模态测量稀疏、不规则且含噪声（质量控制瑕疵），但跨模态间存在相关性；可用模态随空间和时间变化，若模型无法适应任意子集，则有效记录大幅缩减。我们提出OmniField，一种具备连续性感知能力的框架，通过可用模态条件化学习连续神经场，并迭代融合跨模态上下文。采用多模态交叉干扰块架构与迭代跨模态优化，在解码器前对齐信号，实现统一重建、插值、预测和跨模态推断，无需网格化或代理预处理。大量评估表明，OmniField始终优于8个强基线模型。在强模拟传感器噪声下，性能仍接近无噪声输入水平，凸显其对损坏测量的鲁棒性。
原文摘要 · Abstract (English)
Multimodal spatiotemporal learning on real-world experimental data is constrained by two challenges: within-modality measurements are sparse, irregular, and noisy (QA/QC artifacts) but cross-modally correlated; the set of available modalities varies across space and time, shrinking the usable record unless models can adapt to arbitrary subsets at train and test time. We propose OmniField, a continuity-aware framework that learns a continuous neural field conditioned on available modalities and iteratively fuses cross-modal context. A multimodal crosstalk block architecture paired with iterative cross-modal refinement aligns signals prior to the decoder, enabling unified reconstruction, interpolation, forecasting, and cross-modal prediction without gridding or surrogate preprocessing. Extensive evaluations show that OmniField consistently outperforms eight strong multimodal spatiotemporal baselines. Under heavy simulated sensor noise, performance remains close to clean-input levels, highlighting robustness to corrupted measurements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。