用大模型分析帕金森患者自述，自动识别认知状态变化。
Toward Automated Cognitive Assessment in Parkinson's Disease Using Pretrained Language Models
- 用Bio_ClinicalBERT和Llama3等模型识别患者叙述中的认知类别。
- 微调后的Llama3在7类认知任务中平均F1达0.74（微平均）。
- 适合临床研究者做无创长期认知监测，补充传统评估。
了解帕金森病（PD）患者在日常生活中描述认知体验的方式，可为疾病相关的认知与情绪变化提供宝贵见解。然而，从非结构化患者叙述中提取此类信息具有挑战性，因认知概念微妙且重叠。本研究开发并评估了自然语言处理（NLP）模型，用于从去标识化第一人称叙述中自动识别反映多种认知过程的七类范畴。比较了三种模型：基于Bio_ClinicalBERT的嵌套实体识别模型、使用QLoRA微调的Meta-Llama-3-8B-Instruct模型，以及在零样本和少样本设置下评估的GPT-4o mini。结果显示，模型表现因类别和模型家族而异。微调后的Llama3取得最高整体性能（微平均F1=0.74，宏平均F1=0.59），尤其在依赖上下文的类别如思维与社交互动中表现突出。Bio_ClinicalBERT虽精度高但召回率低，部分类别（如地点、时间）表现接近Llama，但在思维、情绪与社交互动上表现不佳。相较于传统信息抽取任务，该任务因叙述中复杂认知过程的抽象性和重叠性更具挑战性。尽管如此，经持续优化后，这些NLP系统有望实现低负担、纵向的认知功能监测，并作为正式神经心理学评估的有力补充。
原文摘要 · Abstract (English)
Understanding how individuals with Parkinson's disease (PD) describe cognitive experiences in their daily lives can offer valuable insights into disease-related cognitive and emotional changes. However, extracting such information from unstructured patient narratives is challenging due to the subtle, overlapping nature of cognitive constructs. This study developed and evaluated natural language processing (NLP) models to automatically identify categories that reflect various cognitive processes from de-identified first-person narratives. Three model families, a Bio_ClinicalBERT-based span categorization model for nested entity recognition, a fine-tuned Meta-Llama-3-8B-Instruct model using QLoRA for instruction following, and GPT-4o mini evaluated under zero- and few-shot settings, were compared on their performance on extracting seven categories. Our findings indicated that model performance varied substantially across categories and model families. The fine-tuned Meta-Llama-3-8B-Instruct achieved the highest overall F1-scores (0.74 micro-average and 0.59 macro-average), particularly excelling in context-dependent categories such as thought and social interaction. Bio_ClinicalBERT exhibited high precision but low recall and performed comparable to Llama for some category types such as location and time but failed on other categories such as thought, emotion and social interaction. Compared to conventional information extraction tasks, this task presents a greater challenge due to the abstract and overlapping nature of narrative accounts of complex cognitive processes. Nonetheless, with continued refinement, these NLP systems hold promise for enabling low-burden, longitudinal monitoring of cognitive function and serving as a valuable complement to formal neuropsychological assessments in PD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。