首个在线灵性交流多模态数据集，助力抽象概念分类研究
SACRED: A Faithful Annotated Multimedia Multimodal Multilingual Dataset for Classifying Connectedness Types in Online Spirituality
- 构建高质量多模态灵性数据集SACRED，确保分类忠实度
- DeepSeek-V3在文本分类达79.19%准确率，GPT-4o-mini视觉任务F1达63.99%
- 发现新型连接关系，适合宗教社会学与跨文化沟通研究
在宗教与神学研究中，灵性因超越文化边界并赋予个体独特体验而备受关注。然而，社会科学家常依赖有限且难以获取的在线数据集。本研究与社会科学家合作，构建了高质量多媒体多模态数据集SACRED，确保分类结果的忠实性。利用SACRED，我们评估了13种主流大模型及传统规则与微调方法的表现。结果显示，DeepSeek-V3在文本分类任务中表现优异（Quora测试集准确率达79.19%），GPT-4o-mini在视觉任务中领先（F1得分为63.99%）。本研究首次构建了来自在线灵性交流的标注多模态数据集，并发现一种新的连接类型，对传播科学具有重要价值。
原文摘要 · Abstract (English)
In religion and theology studies, spirituality has garnered significant research attention for the reason that it not only transcends culture but offers unique experience to each individual. However, social scientists often rely on limited datasets, which are basically unavailable online. In this study, we collaborated with social scientists to develop a high-quality multimedia multi-modal datasets, \textbf{SACRED}, in which the faithfulness of classification is guaranteed. Using \textbf{SACRED}, we evaluated the performance of 13 popular LLMs as well as traditional rule-based and fine-tuned approaches. The result suggests DeepSeek-V3 model performs well in classifying such abstract concepts (i.e., 79.19\% accuracy in the Quora test set), and the GPT-4o-mini model surpassed the other models in the vision tasks (63.99\% F1 score). Purportedly, this is the first annotated multi-modal dataset from online spirituality communication. Our study also found a new type of connectedness which is valuable for communication science studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。