针对僧伽罗语低资源场景，提出新微调策略提升仇恨语言检测性能。
Subasa - Adapting Language Models for Low-resourced Offensive Language Detection in Sinhala
- 引入中间预微调阶段，结合掩码推理预测提升模型表现
- Subasa-XLM-R在基准上达0.84宏F1，超越GPT-4o零样本表现
- 开源模型与代码，适合低资源语言研究者使用
准确识别仇恨语言对社交媒体安全至关重要。高资源与低资源语言在此任务上的表现差异显著。本文首次探索适用于僧伽罗语的新型微调策略，用于仇恨语言检测下游任务。提出四个模型：'Subasa-XLM-R' 在XLM-R基础上增加中间掩码推理预测预微调步骤；'Subasa-Llama' 和 'Subasa-Mistral' 分别基于 Llama (3.2) 与 Mistral (v0.3) 进行任务特异性微调。在僧伽罗语仇恨语言检测基准数据集 SOLD 上评估，所有模型均优于现有基线。其中，Subasa-XLM-R 达到最高宏 F1 分数 0.84，在相同 SOLD 数据集上以零样本设置超越 GPT-4o 等先进大模型。模型与代码已公开。
原文摘要 · Abstract (English)
Accurate detection of offensive language is essential for a number of applications related to social media safety. There is a sharp contrast in performance in this task between low and high-resource languages. In this paper, we adapt fine-tuning strategies that have not been previously explored for Sinhala in the downstream task of offensive language detection. Using this approach, we introduce four models: "Subasa-XLM-R", which incorporates an intermediate Pre-Finetuning step using Masked Rationale Prediction. Two variants of "Subasa-Llama" and "Subasa-Mistral", are fine-tuned versions of Llama (3.2) and Mistral (v0.3), respectively, with a task-specific strategy. We evaluate our models on the SOLD benchmark dataset for Sinhala offensive language detection. All our models outperform existing baselines. Subasa-XLM-R achieves the highest Macro F1 score (0.84) surpassing state-of-the-art large language models like GPT-4o when evaluated on the same SOLD benchmark dataset under zero-shot settings. The models and code are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。