用视频和健康数据精准预测肺功能，低功耗模型适合居家监测
PulmoFusion: Advancing Pulmonary Health with Efficient Multi-Modal Fusion
- 融合热成像、RGB视频与患者信息，用脉冲神经网络做肺功能预测
- 热成像模型呼吸周期准确率达92%，患者级准确率99.5%，肺活量预测误差仅4.52%
- 轻量化设计适合边缘设备，适合远程肺病管理与慢病长期监测
传统远程肺功能检测精度不足。本文提出一种新型非侵入式多模态方法，结合RGB或热成像视频与患者元数据，利用低功耗脉冲神经网络（SNN）回归峰值呼气流速（PEF），并分类强制呼气量（FEV1）与用力肺活量（FVC）。为克服SNN在回归任务中的局限，采用轻量级CNN增强性能。通过多头注意力层提升多模态融合效果，并使用K折交叉验证与集成学习提高模型鲁棒性。基于热成像数据，SNN模型在呼吸周期层面达到92%准确率，患者层面达99.5%。PEF回归相对均方根误差为0.11（热成像）和0.26（RGB），FEV1/FVC预测平均绝对误差为4.52%，性能达当前最优。代码与数据集见https://github.com/ahmed-sharshar/RespiroDynamics.git。
原文摘要 · Abstract (English)
Traditional remote spirometry lacks the precision required for effective pulmonary monitoring. We present a novel, non-invasive approach using multimodal predictive models that integrate RGB or thermal video data with patient metadata. Our method leverages energy-efficient Spiking Neural Networks (SNNs) for the regression of Peak Expiratory Flow (PEF) and classification of Forced Expiratory Volume (FEV1) and Forced Vital Capacity (FVC), using lightweight CNNs to overcome SNN limitations in regression tasks. Multimodal data integration is improved with a Multi-Head Attention Layer, and we employ K-Fold validation and ensemble learning to boost robustness. Using thermal data, our SNN models achieve 92% accuracy on a breathing-cycle basis and 99.5% patient-wise. PEF regression models attain Relative RMSEs of 0.11 (thermal) and 0.26 (RGB), with an MAE of 4.52% for FEV1/FVC predictions, establishing state-of-the-art performance. Code and dataset can be found on https://github.com/ahmed-sharshar/RespiroDynamics.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。