将可穿戴设备的运动数据转化为自然语言摘要,让机器读懂你的日常行为。
MotionTeller: Multi-modal Integration of Wearable Time-Series with LLMs for Health and Behavioral Understanding
- 用预训练的活动编码器+轻量投影模块,把生理信号映射到大模型的词空间中
- 在54383条真实数据上训练,生成文本的语义准确率高达92.4%,比基线高7%
- 适合临床评估、健康监测和个性化干预,输出结果自然流畅且可解释
随着可穿戴传感器日益普及,一个核心挑战是:如何从原始生理信号(如分钟级加速度计记录的活动数据)生成自然语言摘要?本文提出MotionTeller,一种原生融合分钟级可穿戴活动数据与大语言模型(LLM)的生成框架。该模型结合预训练的活动编码器与轻量级投影模块,将行为嵌入映射至冻结的解码器型LLM的词空间,实现自回归的自由文本生成。我们构建了一个包含54383对(活动数据,文本)的真实世界NHANES数据集,仅对语言标记进行交叉熵损失监督训练。MotionTeller在语义一致性(BERTScore-F1 = 0.924)和词汇准确性(ROUGE-1 = 0.722)上表现优异,相较提示工程基线提升7个百分点。平均训练损失在第15轮收敛至0.38,表明优化稳定。定性分析显示模型捕捉了昼夜节律结构与行为转换,主成分分析图显示训练后嵌入空间聚类对齐更优。这些结果表明MotionTeller是一种可扩展、可解释的系统,能将可穿戴传感数据转化为流畅的人类中心描述,为行为监测、临床审查与个性化健康干预开辟新路径。
原文摘要 · Abstract (English)
As wearable sensing becomes increasingly pervasive, a key challenge remains: how can we generate natural language summaries from raw physiological signals such as actigraphy - minute-level movement data collected via accelerometers? In this work, we introduce MotionTeller, a generative framework that natively integrates minute-level wearable activity data with large language models (LLMs). MotionTeller combines a pretrained actigraphy encoder with a lightweight projection module that maps behavioral embeddings into the token space of a frozen decoder-only LLM, enabling free-text, autoregressive generation of daily behavioral summaries. We construct a novel dataset of 54383 (actigraphy, text) pairs derived from real-world NHANES recordings, and train the model using cross-entropy loss with supervision only on the language tokens. MotionTeller achieves high semantic fidelity (BERTScore-F1 = 0.924) and lexical accuracy (ROUGE-1 = 0.722), outperforming prompt-based baselines by 7 percent in ROUGE-1. The average training loss converges to 0.38 by epoch 15, indicating stable optimization. Qualitative analysis confirms that MotionTeller captures circadian structure and behavioral transitions, while PCA plots reveal enhanced cluster alignment in embedding space post-training. Together, these results position MotionTeller as a scalable, interpretable system for transforming wearable sensor data into fluent, human-centered descriptions, introducing new pathways for behavioral monitoring, clinical review, and personalized health interventions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。