用智能穿戴+大模型实现远程健康自动监控,能识情绪、判跌倒、主动告警。
REMONI: An Autonomous System Integrating Wearables and Multimodal Large Language Models for Enhanced Remote Health Monitoring
- 融合可穿戴设备、多模态大模型与物联网,自动采集并分析生理数据与视频。
- 支持跌倒检测与紧急情况实时告警,医生可通过自然语言交互获取患者状态。
- 采用提示工程整合信息,适合医疗监护、居家养老等场景应用。
随着可穿戴设备的普及,远程患者监测需求日益增长。现有研究多聚焦于传感器数据采集、可视化与异常分析,但人机交互环节存在明显短板。本文提出REMONI系统,一个集成多模态大语言模型(MLLMs)、物联网(IoT)与可穿戴设备的自主远程健康监测系统。系统持续自动采集智能手表等设备的体征数据、加速度数据及摄像头拍摄的患者视频片段,通过异常检测模块(含跌倒检测模型)识别紧急状况并通知照护人员。系统独特之处在于基于MLLM的自然语言处理组件,可识别患者活动与情绪,并响应医护人员提问。同时,借助提示工程实现患者信息无缝整合。医生与护士可通过友好的网页应用,以自然语言交互方式实时获取患者生命体征与当前状态、情绪。实验表明,该系统在真实场景中具备可行性与可扩展性,有望减轻医护人员负担并降低医疗成本。已开发完整原型并进行测试,验证了各项功能的鲁棒性。
原文摘要 · Abstract (English)
With the widespread adoption of wearable devices in our daily lives, the demand and appeal for remote patient monitoring have significantly increased. Most research in this field has concentrated on collecting sensor data, visualizing it, and analyzing it to detect anomalies in specific diseases such as diabetes, heart disease and depression. However, this domain has a notable gap in the aspect of human-machine interaction. This paper proposes REMONI, an autonomous REmote health MONItoring system that integrates multimodal large language models (MLLMs), the Internet of Things (IoT), and wearable devices. The system automatically and continuously collects vital signs, accelerometer data from a special wearable (such as a smartwatch), and visual data in patient video clips collected from cameras. This data is processed by an anomaly detection module, which includes a fall detection model and algorithms to identify and alert caregivers of the patient's emergency conditions. A distinctive feature of our proposed system is the natural language processing component, developed with MLLMs capable of detecting and recognizing a patient's activity and emotion while responding to healthcare worker's inquiries. Additionally, prompt engineering is employed to integrate all patient information seamlessly. As a result, doctors and nurses can access real-time vital signs and the patient's current state and mood by interacting with an intelligent agent through a user-friendly web application. Our experiments demonstrate that our system is implementable and scalable for real-life scenarios, potentially reducing the workload of medical professionals and healthcare costs. A full-fledged prototype illustrating the functionalities of the system has been developed and being tested to demonstrate the robustness of its various capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。