构建实时检测越南地域歧视言论的系统，填补中文语料空白
A Big Data-empowered System for Real-time Detection of Regional Discriminatory Comments on Vietnamese Social Media
- 基于机器学习与迁移学习，构建越南地域歧视评论检测任务
- 创建包含社交平台评论的ViRDC数据集，支持实时分析
- 基于Apache Spark实现实时流处理，适合政策监管与平台风控
地域歧视是越南长期存在的社会问题。尽管已有研究关注越南语仇恨言论，但针对地域歧视的研究仍显不足，且多聚焦于模型开发而忽略实际系统部署。本文提出在越南社交媒体上检测地域歧视评论的任务，利用机器学习与迁移学习技术。我们构建了包含社交平台评论的ViRDC数据集，为后续研究提供重要资源。系统集成流式处理能力，通过Apache Spark框架实现对社交媒体数据的实时高效处理，确保可扩展性与响应速度。该系统为越南地区实时识别地域歧视言论提供了完整解决方案。
原文摘要 · Abstract (English)
Regional discrimination is a persistent social issue in Vietnam. While existing research has explored hate speech in the Vietnamese language, the specific issue of regional discrimination remains under-addressed. Previous studies primarily focused on model development without considering practical system implementation. In this work, we propose a task called Detection of Regional Discriminatory Comments on Vietnamese Social Media, leveraging the power of machine learning and transfer learning models. We have built the ViRDC (Vietnamese Regional Discrimination Comments) dataset, which contains comments from social media platforms, providing a valuable resource for further research and development. Our approach integrates streaming capabilities to process real-time data from social media networks, ensuring the system's scalability and responsiveness. We developed the system on the Apache Spark framework to efficiently handle increasing data inputs during streaming. Our system offers a comprehensive solution for the real-time detection of regional discrimination in Vietnam.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。