通过融合标注者信息与损失重加权，提升对标注分歧的预测能力。
LPI-RIT at LeWiDi-2025: Improving Distributional Predictions via Metadata and Loss Reweighting with DisCo
- 引入标注者元数据嵌入，增强输入表征以捕捉分歧模式。
- 多目标训练损失使软标签分布预测在三组数据上均有显著提升。
- 适合关注标注一致性与主观评价建模的研究者参考。
LeWiDi-2025 共享任务旨在通过软标签分布预测和视角评估来建模标注者分歧，重点关注个体标注者。我们基于 DisCo（从上下文推断分布）这一神经架构，提出改进方案并进行深入分析。本文通过引入标注者元数据嵌入、增强输入表示，并采用多目标训练损失，进一步优化了项目级与标注者级标签分布的联合建模。在三个数据集上的大量实验表明，软标签预测与视角评估指标均取得显著提升。深入的校准与错误分析揭示了分歧感知建模在何种条件下有效。结果表明，结合标注者人口统计学特征并直接优化分布指标，可更准确捕捉分歧模式，在不同数据集上均带来稳定改进。
原文摘要 · Abstract (English)
The Learning With Disagreements (LeWiDi) 2025 shared task aims to model annotator disagreement through soft label distribution prediction and perspectivist evaluation, which focuses on modeling individual annotators. We adapt DisCo (Distribution from Context), a neural architecture that jointly models item-level and annotator-level label distributions, and present detailed analysis and improvements. In this paper, we extend DisCo by introducing annotator metadata embeddings, enhancing input representations, and multi-objective training losses to capture disagreement patterns better. Through extensive experiments, we demonstrate substantial improvements in both soft and perspectivist evaluation metrics across three datasets. We also conduct in-depth calibration and error analyses that reveal when and why disagreement-aware modeling improves. Our findings show that disagreement can be better captured by conditioning on annotator demographics and by optimizing directly for distributional metrics, yielding consistent improvements across datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。