用神经算子融合影像与数据，实时预测脑损伤时的变形情况。
Multimodal Neural Operators for Real-Time Biomechanical Modelling of Traumatic Brain Injury
- 将脑部影像、人口统计和采集参数作为多模态输入，构建神经算子模型。
- DeepONet在真实位移场上误差最小（MSE=0.0039），推理速度最快（3.83帧/秒）。
- 不同架构各有优劣，适合不同临床需求，为生物医学建模提供选型参考。
背景：创伤性脑损伤建模需融合体素化神经影像、人口统计参数和采集元数据。有限元求解器计算成本过高，难以用于临床。神经算子可大幅加速推理，但其在整合体素影像与标量元数据方面的潜力尚未充分探索。目标：评估多模态神经算子架构在脑生物力学建模中的表现。测试了融合体素解剖影像、人口统计特征与采集参数以预测来自MRE数据的全场脑位移的方法。方法：将TBI建模视为多模态算子学习问题。采用两种融合策略：傅里叶神经算子（FNO）使用场投影，深度算子网络（DeepONet）采用分支分解。在249个体内MRE数据集（频率20–90 Hz）上评估了四种模型（FNO、因子化FNO、多网格FNO、DeepONet）。结果：DeepONet在真实位移场上表现最佳（MSE=0.0039，准确率90.0%），推理最快（3.83 it/s），参数最少（2.09M）。MG-FNO在虚部场表现最优（MSE=0.0058，准确率88.3%），所需GPU内存最低（7.12 GB）。无单一架构在所有指标上占优，揭示了精度、空间保真度与计算成本间的权衡。结论：结合多模态融合的神经算子能准确预测异构输入下的全场脑位移，推理速度比有限元求解器快数个数量级。本比较为生物医学场景中算子学习方法的选择提供了指导。
原文摘要 · Abstract (English)
Background: Traumatic brain injury modeling requires integrating volumetric neuroimaging, demographic parameters, and acquisition metadata. Finite element solvers are too computationally expensive for clinical settings. Neural operators offer much faster inference. Their ability to integrate volumetric imaging with scalar metadata remains underexplored for biomechanical predictions. Objective: This study evaluates multimodal neural operator architectures for brain biomechanics. We test strategies fusing volumetric anatomical imaging, demographic features, and acquisition parameters to predict full-field brain displacement from MRE data. Methods: We framed TBI modeling as a multimodal operator learning problem. Two fusion strategies were tested. Field projection was applied for Fourier Neural Operator (FNO) architectures. Branch decomposition was used for Deep Operator Networks (DeepONet). Four models (FNO, Factorized FNO, Multi-Grid FNO, DeepONet) were evaluated on 249 in vivo MRE datasets across frequencies from 20 to 90 Hz. Results: DeepONet achieved the highest accuracy on real displacement fields (MSE = 0.0039, 90.0% accuracy) with the fastest inference (3.83 it/s) and fewest parameters (2.09M). MG-FNO performed best on imaginary fields (MSE = 0.0058, 88.3% accuracy) requiring the lowest GPU memory among FNO variants (7.12 GB). No single architecture dominated all criteria. This reveals distinct trade-offs between accuracy, spatial fidelity, and computational cost. Conclusion: Neural operators augmented with multimodal fusion can accurately predict full-field brain displacement from heterogeneous inputs. They offer inference times orders of magnitude faster than finite element solvers. This comparison provides guidance for selecting operator learning approaches in biomedical settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。