arXiv:2507.10996cs.CL2025-07被引 6

用轻量微调实现中英文性别歧视文本的精准识别。

Mario at EXIST 2025: A Simple Gateway to Effective Multilingual Sexism Detection

  • 通过分层适配器路由,统一处理三类子任务。
  • 仅用1.67%参数量,训练速度提升75%,性能接近全量微调。
  • 无需多语言模型,跨语言迁移带来1.7%-2.4%性能提升。

本文介绍我们在EXIST 2025任务1中的方法,针对英语和西班牙语推文的文本性别歧视检测,采用Llama 3.1 8B的分层低秩适配(LoRA)。提出条件适配器路由,显式建模三个层级子任务间的标签依赖关系:二分类性别歧视识别、来源意图判定、多标签性别歧视分类。不同于仅适配注意力层的传统LoRA,本方法对所有线性变换进行适配,增强模型捕捉任务特异性模式的能力。相比复杂的数据处理与集成方法,我们证明简单高效的参数高效微调可取得优异表现。为每个子任务分别训练独立的LoRA适配器(秩=16,QLoRA 4位),采用统一多语言训练,利用Llama 3.1原生双语能力。方法仅需最少预处理,使用标准监督学习。多语言训练策略避免了语言专用模型需求,通过跨语言迁移实现1.7%-2.4%的F1提升。相比全量微调,仅1.67%可训练参数,训练时间减少75%,模型存储降低98%,在所有子任务上均达竞争性性能(二分类ICM-Hard: 0.6774,意图检测: 0.4991,多标签分类: 0.6519)。

原文摘要 · Abstract (English)

This paper presents our approach to EXIST 2025 Task 1, addressing text-based sexism detection in English and Spanish tweets through hierarchical Low-Rank Adaptation (LoRA) of Llama 3.1 8B. Our method introduces conditional adapter routing that explicitly models label dependencies across three hierarchically structured subtasks: binary sexism identification, source intention detection, and multilabel sexism categorization. Unlike conventional LoRA applications that target only attention layers, we apply adaptation to all linear transformations, enhancing the model's capacity to capture task-specific patterns. In contrast to complex data processing and ensemble approaches, we show that straightforward parameter-efficient fine-tuning achieves strong performance. We train separate LoRA adapters (rank=16, QLoRA 4-bit) for each subtask using unified multilingual training that leverages Llama 3.1's native bilingual capabilities. The method requires minimal preprocessing and uses standard supervised learning. Our multilingual training strategy eliminates the need for separate language-specific models, achieving 1.7-2.4\% F1 improvements through cross-lingual transfer. With only 1.67\% trainable parameters compared to full fine-tuning, our approach reduces training time by 75\% and model storage by 98\%, while achieving competitive performance across all subtasks (ICM-Hard: 0.6774 for binary classification, 0.4991 for intention detection, 0.6519 for multilabel categorization).

多语言性别歧视检测轻量微调LoRA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。