用大模型分析地铁社交数据,自动识别问题与情绪。
Transit Pulse: Utilizing Social Media as a Source for Customer Feedback and Information Extraction with Large Language Model
- 用Llama 3结合RAG技术,无需预设标签提取信息
- 在真实地铁数据上准确识别情绪、讽刺与异常事件
- 适合交通部门快速响应市民反馈,提升服务
公共交通用户每日在社交网络发布大量信息,蕴含改进服务的关键洞察。这些内容帮助机构快速发现新问题。主题与情感解析是获取全面见解的核心,但消息量庞大,传统NLP方法如TF-IDF难以处理语义细微差别。现有情感分析将主题与情绪分离,忽略二者交互,影响分类效率。为此,本文提出一种新方法,利用大语言模型(LLM)——Llama 3,实现无需预设标签的流畅分析,支持情感、讽刺检测、异常系统问题识别及位置信息提取。通过引入检索增强生成(RAG),融合外部知识源以增强领域理解。我们在真实地铁系统的用户推文数据上进行了广泛实验,结果表明,该方法显著优于传统NLP技术,可从社交媒体中提取更丰富、更及时的可用信息,有效提升公交机构的响应能力。
原文摘要 · Abstract (English)
Users of the transit system flood social networks daily with messages that contain valuable insights crucial for improving service quality. These posts help transit agencies quickly identify emerging issues. Parsing topics and sentiments is key to gaining comprehensive insights to foster service excellence. However, the volume of messages makes manual analysis impractical, and standard NLP techniques like Term Frequency-Inverse Document Frequency (TF-IDF) fall short in nuanced interpretation. Traditional sentiment analysis separates topics and sentiments before integrating them, often missing the interaction between them. This incremental approach complicates classification and reduces analytical productivity. To address these challenges, we propose a novel approach to extracting and analyzing transit-related information, including sentiment and sarcasm detection, identification of unusual system problems, and location data from social media. Our method employs Large Language Models (LLM), specifically Llama 3, for a streamlined analysis free from pre-established topic labels. To enhance the model's domain-specific knowledge, we utilize Retrieval-Augmented Generation (RAG), integrating external knowledge sources into the information extraction pipeline. We validated our method through extensive experiments comparing its performance with traditional NLP approaches on user tweet data from the real world transit system. Our results demonstrate the potential of LLMs to transform social media data analysis in the public transit domain, providing actionable insights and enhancing transit agencies' responsiveness by extracting a broader range of information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。