用词法相似度评估作文，发现余弦相似度更准。
Automatic essay scoring: leveraging Jaccard coefficient and Cosine similaritywith n-gram variation in vector space model approach
- 用一元、二元、三元词组构建向量，计算文本相似度。
- 余弦相似度的评分误差比杰卡德系数低，效果更好。
- 单个词的表示比组合词更有效，适合教学评估。
自动作文评分（AES）是高效准确评估写作内容的重要研究方向。本研究探讨了在向量空间模型（VSM）中，使用一元、二元和三元词组表示时，杰卡德系数与余弦相似度两种相似性度量的有效性。数据来自初中公民教育课程的形成性作文。每篇作文经预处理后，通过n-gram模型提取特征并向量化为数值表示，再分别用杰卡德系数和余弦相似度计算作文间相似度。系统性能通过人类评分与系统评分之间的均方根误差（RMSE）进行评估。结果显示，余弦相似度表现优于杰卡德系数；在n-gram层面，一元词组的RMSE低于二元和三元词组。
原文摘要 · Abstract (English)
Automated essay scoring (AES) is a vital area of research aiming to provide efficient and accurate assessment tools for evaluating written content. This study investigates the effectiveness of two popular similarity metrics, Jaccard coefficient, and Cosine similarity, within the context of vector space models(VSM)employing unigram, bigram, and trigram representations. The data used in this research was obtained from the formative essay of the citizenship education subject in a junior high school. Each essay undergoes preprocessing to extract features using n-gram models, followed by vectorization to transform text data into numerical representations. Then, similarity scores are computed between essays using both Jaccard coefficient and Cosine similarity. The performance of the system is evaluated by analyzing the root mean square error (RMSE), which measures the difference between the scores given by human graders and those generated by the system. The result shows that the Cosine similarity outperformed the Jaccard coefficient. In terms of n-gram, unigrams have lower RMSE compared to bigrams and trigrams.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。