arXiv:2410.16924cs.AI2024-10被引 5

小模型通过思维链蒸馏,实现个性化睡眠健康推荐。

SleepCoT: A Lightweight Personalized Sleep Health Model via Chain-of-Thought Distillation

  • 用思维链蒸馏将大模型能力迁移到小模型,提升推理与知识应用。
  • 在100份模拟报告和1000个问题上表现接近大模型,效率更高。
  • 适合资源受限场景,推动个性化医疗普惠化。

我们提出一种基于少样本思维链(CoT)蒸馏的个性化睡眠健康管理新方法,使参数量大于20亿的小型语言模型在专业健康领域表现媲美大型语言模型(LLMs)。该方法同时将问题求解策略、长尾专家知识及个性化推荐能力从大模型中蒸馏到更高效紧凑的模型中。与现有系统不同,本方法具备三项核心功能:生成个性化睡眠建议、支持用户特定追问、回答领域专有知识问题。研究聚焦于睡眠健康,因其可通过可穿戴设备测量且显著影响整体健康。实验采用GPT-4o进行数据合成,Qwen-max构建指令集,Qwen2.5 1.5B执行模型蒸馏。在100份模拟睡眠报告和1,000个领域特定问题上的测试表明,该模型在惩罚度、推理能力和知识应用方面显著优于基线小型模型,性能接近大模型,同时保持部署效率。本研究不仅推进了人工智能驱动的健康管理,还为在资源受限环境中利用大模型能力提供了新路径,有望提升个性化医疗的可及性。

原文摘要 · Abstract (English)

We present a novel approach to personalized sleep health management using few-shot Chain-of-Thought (CoT) distillation, enabling small-scale language models (> 2B parameters) to rival the performance of large language models (LLMs) in specialized health domains. Our method simultaneously distills problem-solving strategies, long-tail expert knowledge, and personalized recommendation capabilities from larger models into more efficient, compact models. Unlike existing systems, our approach offers three key functionalities: generating personalized sleep health recommendations, supporting user-specific follow-up inquiries, and providing responses to domain-specific knowledge questions. We focus on sleep health due to its measurability via wearable devices and its impact on overall well-being. Our experimental setup, involving GPT-4o for data synthesis, Qwen-max for instruction set creation, and Qwen2.5 1.5B for model distillation, demonstrates significant improvements over baseline small-scale models in penalization, reasoning, and knowledge application. Experiments using 100 simulated sleep reports and 1,000 domain-specific questions shows our model achieves comparable performance to larger models while maintaining efficiency for real-world deployment. This research not only advances AI-driven health management but also provides a novel approach to leveraging LLM capabilities in resource-constrained environments, potentially enhancing the accessibility of personalized healthcare solutions.

睡眠健康小模型思维链个性化医疗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。