对比了多种模型在六类心理疾病分类中的表现,发现Transformer更准但LSTM更快。
Mental Multi-class Classification on Social Media: Benchmarking Transformer Architectures against LSTM Models
- 用Reddit数据构建六类心理疾病+对照组的大规模标注集
- RoBERTa模型在所有类别上达到91%-99%的F1分数,最优
- 带注意力机制的LSTM+BERT嵌入可接近Transformer性能且快2-3.5倍
数百万用户在社交媒体上公开分享心理健康问题,为抑郁、双相情感障碍等疾病的早期检测提供了丰富数据。然而,以往自然语言处理研究多集中于单一疾病识别,缺乏对多类心理疾病区分能力的系统评估。本文开展大规模对比研究,比较最先进的Transformer模型(BERT、RoBERTa、DistilBERT、ALBERT、ELECTRA)与多种LSTM变体(含/不含注意力机制,使用上下文或静态嵌入)在心理状态多分类任务中的表现。研究首先基于严格的过滤和统计分析,构建了一个涵盖六类心理疾病及对照组的大型Reddit数据集。实验结果显示,所有Transformer模型均显著优于基准模型,其中RoBERTa在各类别上实现91%-99%的F1分数和准确率。值得注意的是,使用BERT嵌入并引入注意力机制的LSTM模型性能接近变压器(最高达97% F1分数),同时训练速度提升2-3.5倍;而使用静态嵌入的LSTM无法学习有效特征。该研究首次提供多类心理疾病检测的全面基准,为实际部署中的模型选择提供依据,并揭示精度与效率间的权衡关系。
原文摘要 · Abstract (English)
Millions of people openly share mental health struggles on social media, providing rich data for early detection of conditions such as depression, bipolar disorder, etc. However, most prior Natural Language Processing (NLP) research has focused on single-disorder identification, leaving a gap in understanding the efficacy of advanced NLP techniques for distinguishing among multiple mental health conditions. In this work, we present a large-scale comparative study of state-of-the-art transformer versus Long Short-Term Memory (LSTM)-based models to classify mental health posts into exclusive categories of mental health conditions. We first curate a large dataset of Reddit posts spanning six mental health conditions and a control group, using rigorous filtering and statistical exploratory analysis to ensure annotation quality. We then evaluate five transformer architectures (BERT, RoBERTa, DistilBERT, ALBERT, and ELECTRA) against several LSTM variants (with or without attention, using contextual or static embeddings) under identical conditions. Experimental results show that transformer models consistently outperform the alternatives, with RoBERTa achieving 91-99% F1-scores and accuracies across all classes. Notably, attention-augmented LSTMs with BERT embeddings approach transformer performance (up to 97% F1-score) while training 2-3.5 times faster, whereas LSTMs using static embeddings fail to learn useful signals. These findings represent the first comprehensive benchmark for multi-class mental health detection, offering practical guidance on model selection and highlighting an accuracy-efficiency trade-off for real-world deployment of mental health NLP systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。