用大模型分析非裔美国人健康故事，提炼出26个关键主题
Topic-aware Large Language Models for Summarizing the Lived Healthcare Experiences Described in Health Stories
- 结合LDA与开源大模型，分层总结非裔美国人健康叙事
- 生成的主题摘要无虚构、准确且全面，获专家认可
- 适合医疗研究者挖掘患者真实体验，优化干预策略
讲故事是强有力的沟通方式，可能揭示影响医疗结果差距的深层因素。为检验大语言模型（LLM）识别潜在原因和干预路径的能力，我们对50篇非裔美国人（AA）叙述进行了主题感知的层级化摘要。使用隐含狄利克雷分配（LDA）技术从这些故事中识别出26个主题，并基于开源LLM的层级摘要方法对每类主题下的故事进行摘要。通过将各故事摘要整合生成主题摘要，并由GPT4模型评估其虚构性、准确性、完整性和实用性，评估结果经两位领域专家验证可靠性。结果显示，主题摘要无虚构内容，具备高度准确性、完整性和实用性；GPT4评分与专家评估呈现中等到高等度一致性。该方法成功提取了与非裔美国人经验相关的关键主题，如健康行为、与医疗团队互动、照护与症状管理等。这些洞察可帮助研究者高效学习非结构化叙述，发现潜在影响因素与干预点，提升患者与照护者支持水平，最终改善健康结局。
原文摘要 · Abstract (English)
Storytelling is a powerful form of communication and may provide insights into factors contributing to gaps in healthcare outcomes. To determine whether Large Language Models (LLMs) can identify potential underlying factors and avenues for intervention, we performed topic-aware hierarchical summarization of narratives from African American (AA) storytellers. Fifty transcribed stories of AA experiences were used to identify topics in their experience using the Latent Dirichlet Allocation (LDA) technique. Stories about a given topic were summarized using an open-source LLM-based hierarchical summarization approach. Topic summaries were generated by summarizing across story summaries for each story that addressed a given topic. Generated topic summaries were rated for fabrication, accuracy, comprehensiveness, and usefulness by the GPT4 model, and the model's reliability was validated against the original story summaries by two domain experts. 26 topics were identified in the fifty AA stories. The GPT4 ratings suggest that topic summaries were free from fabrication, highly accurate, comprehensive, and useful. The reliability of GPT ratings compared to expert assessments showed moderate to high agreement. Our approach identified AA experience-relevant topics such as health behaviors, interactions with medical team members, caregiving and symptom management, among others. Such insights could help researchers identify potential factors and interventions by learning from unstructured narratives in an efficient manner-leveraging the communicative power of storytelling. The use of LDA and LLMs to identify and summarize the experience of AA individuals suggests a variety of possible avenues for health research and possible clinical improvements to support patients and caregivers, thereby ultimately improving health outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。