构建多语言推文话题分类数据集，助力跨语言内容分析
Multilingual Topic Classification in X: Dataset and Analysis
- 构建涵盖英西日希四语的推特话题数据集X-Topic
- 验证主流语言模型在多语言场景下的分类能力差异
- 适合从事跨语言分析与多模态模型开发的研究者
在社交媒体动态环境中，每日都有跨越语言边界的多样化话题讨论。然而，传统方法如主题建模在处理多语言内容时仍面临挑战。本文提出X-Topic，一个包含英语、西班牙语、日语和希腊语四种语言的多语言推文话题分类数据集，覆盖广泛社交内容主题，可为跨语言分析、鲁棒多语言模型开发及在线对话研究提供重要资源。我们利用该数据集开展全面的跨语言与多语言分析，比较当前通用型与领域特定语言模型的性能表现。
原文摘要 · Abstract (English)
In the dynamic realm of social media, diverse topics are discussed daily, transcending linguistic boundaries. However, the complexities of understanding and categorising this content across various languages remain an important challenge with traditional techniques like topic modelling often struggling to accommodate this multilingual diversity. In this paper, we introduce X-Topic, a multilingual dataset featuring content in four distinct languages (English, Spanish, Japanese, and Greek), crafted for the purpose of tweet topic classification. Our dataset includes a wide range of topics, tailored for social media content, making it a valuable resource for scientists and professionals working on cross-linguistic analysis, the development of robust multilingual models, and computational scientists studying online dialogue. Finally, we leverage X-Topic to perform a comprehensive cross-linguistic and multilingual analysis, and compare the capabilities of current general- and domain-specific language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。