融合TF-IDF与BERT，提升马拉地语抄袭检测准确率
Enhancing Plagiarism Detection in Marathi with a Weighted Ensemble of TF-IDF and BERT Embeddings for Low-Resource Language Processing
- 用TF-IDF和BERT嵌入加权集成，捕捉文本统计、语义与句法特征
- 通过加权投票机制，在马拉地语数据集上实现更高检测精度
- 适合低资源语言文本分析，尤其关注印度区域性语言的抄袭检测
抄袭指未经适当引用而使用他人作品或观点，并将其作为原创内容呈现。随着印度区域性语言如马拉地语中信息量快速增长，开发针对低资源语言的稳健抄袭检测系统变得至关重要。基于双向编码器表示的Transformer（BERT）等语言模型在文本表征与特征提取方面表现出色，已成为语义分析与抄袭检测的重要工具。然而，其在低资源语言中的应用仍不充分，尤其是在抄袭检测场景下。本文提出一种方法，结合BERT句子嵌入与词频-逆文档频率（TF-IDF）特征表示，提升马拉地语文本的抄袭检测准确率。该方法通过机器学习模型的加权投票集成，有效捕捉文本的统计、语义与句法特征。
原文摘要 · Abstract (English)
Plagiarism involves using another person's work or concepts without proper attribution, presenting them as original creations. With the growing amount of data communicated in regional languages such as Marathi -- one of India's regional languages -- it is crucial to design robust plagiarism detection systems tailored for low-resource languages. Language models like Bidirectional Encoder Representations from Transformers (BERT) have demonstrated exceptional capability in text representation and feature extraction, making them essential tools for semantic analysis and plagiarism detection. However, the application of BERT for low-resource languages remains under-explored, particularly in the context of plagiarism detection. This paper presents a method to enhance the accuracy of plagiarism detection for Marathi texts using BERT sentence embeddings in conjunction with Term Frequency-Inverse Document Frequency (TF-IDF) feature representation. This approach effectively captures statistical, semantic, and syntactic aspects of text features through a weighted voting ensemble of machine learning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。