用NLP方法分类蛋白质序列，发现进化相似性影响模型表现
Protein sequence classification using natural language processing techniques
- 用投票集成与Transformer模型处理蛋白质序列，提升分类效果
- 随机分组下最高准确率77%，但按进化家族分组时性能下降明显
- 使用生物合理划分数据能更真实评估模型泛化能力，适合结构生物学研究
目的：本研究旨在利用自然语言处理（NLP）技术提升蛋白质序列分类性能，并探讨序列相似性对模型表现的影响。比较了多种机器学习与深度学习模型在两种数据分割策略下的表现：随机分割与基于ECOD家族的分割（确保进化相关序列归于同一组）。方法：评估了KNN、朴素贝叶斯、逻辑回归、多层感知机（MLP）、决策树、随机森林、XGBoost、投票与堆叠分类器、卷积神经网络（CNN）、长短期记忆网络（LSTM）以及Transformer模型（BertForSequenceClassification、DistilBERT、ProtBert）。在不同氨基酸范围和序列长度下测试性能，重点关注对未见进化家族的泛化能力。结果：随机分割下，投票分类器达到最高性能，准确率74%、加权F1分数74%、宏平均F1分数65%；在Transformer模型中，ProtBERT表现最佳，准确率77%、加权F1分数76%、宏平均F1分数61%。然而，在采用ECOD家族分割时，所有模型性能均下降，揭示了序列相似性对分类性能的显著影响。结论：先进的NLP技术，特别是集成方法与Transformer模型，在蛋白质分类中展现出巨大潜力，但需充足训练数据并有效管理序列相似性。同时，采用生物意义明确的数据分割方式（如基于ECOD家族）对于真实评估模型泛化能力至关重要。
原文摘要 · Abstract (English)
Purpose: This study aimed to enhance protein sequence classification using natural language processing (NLP) techniques while addressing the impact of sequence similarity on model performance. We compared various machine learning and deep learning models under two different data-splitting strategies: random splitting and ECOD family-based splitting, which ensures evolutionary-related sequences are grouped together. Methods: The study evaluated models such as K-Nearest Neighbors (KNN), Multinomial Naïve Bayes, Logistic Regression, Multi-Layer Perceptron (MLP), Decision Tree, Random Forest, XGBoost, Voting and Stacking classifiers, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and transformer models (BertForSequenceClassification, DistilBERT, and ProtBert). Performance was tested using different amino acid ranges and sequence lengths with a focus on generalization across unseen evolutionary families. Results: The Voting classifier achieved the highest performance with 74% accuracy, 74% weighted F1 score, and 65% macro F1 score under random splitting, while ProtBERT obtained 77% accuracy, 76% weighted F1 score, and 61% macro F1 score among transformer models. However, performance declined across all models when tested using ECOD-based splitting, revealing the impact of sequence similarity on classification performance. Conclusion: Advanced NLP techniques, particularly ensemble methods like Voting classifiers, and transformer models show significant potential in protein classification, with sufficient training data and sequence similarity management being crucial for optimal performance. However, the use of biologically meaningful splitting methods, such as ECOD family-based splitting, is crucial for realistic performance evaluation and generalization to unseen evolutionary families.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。