用无监督聚类分析公众情绪,无需标注数据也能精准捕捉情感分布。
Enhancing Affinity Propagation for Improved Public Sentiment Insights
- 结合凝聚层次聚类优化Affinity Propagation,自动发现情感群体。
- 在多个评估指标上,混合方法显著优于K-means。
- 适合缺乏标注数据的舆情分析场景,如社会热点追踪。
每天生成的海量数据使公众情绪成为营销、政治与社会研究的关键因素。传统情感分析多依赖监督学习,需大量标注数据,成本高且耗时。本文提出一种基于无监督学习的方法,利用亲和传播(Affinity Propagation, AP)聚类分析文本情感。AP通过自然模式分组,无需预设聚类数量。研究采用TF-IDF向量表示文本,结合主成分分析(PCA)降维,并将AP与凝聚层次聚类融合,提升对全局与局部情感结构的捕捉能力。使用轮廓系数(Silhouette Score)、Calinski-Harabasz指数和Davies-Bouldin指数评估性能。实验表明,该混合方法显著优于K-means。本研究为自然语言处理提供了可扩展、高效的无监督情感分析框架,展示了先进AI技术在无需大量标注数据情况下分析公众情绪的社会价值。
原文摘要 · Abstract (English)
With the large amount of data generated every day, public sentiment is a key factor for various fields, including marketing, politics, and social research. Understanding the public sentiment about different topics can provide valuable insights. However, most traditional approaches for sentiment analysis often depend on supervised learning, which requires a significant amount of labeled data. This makes it both expensive and time-consuming to implement. This project introduces an approach using unsupervised learning techniques, particularly Affinity Propagation (AP) clustering, to analyze sentiment. AP clustering groups text data based on natural patterns, without needing predefined cluster numbers. The paper compares AP with K-means clustering, using TF-IDF Vectorization for text representation and Principal Component Analysis (PCA) for dimensionality reduction. To enhance performance, AP is combined with Agglomerative Hierarchical Clustering. This hybrid method refines clusters further, capturing both global and local sentiment structures more effectively. The effectiveness of these methods is evaluated using the Silhouette Score, Calinski-Harabasz Score, and Davies-Bouldin Index. Results show that AP with Agglomerative Hierarchical Clustering significantly outperforms K-means. This research contributes to Natural Language Processing (NLP) by proposing a scalable and efficient unsupervised learning framework for sentiment analysis, highlighting the significant societal impact of advanced AI techniques in analyzing public sentiment without the need for extensive labeled data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。