用人口统计信息增强模型，更好理解标注者分歧。
Modeling Annotator Disagreement with Demographic-Aware Experts and Synthetic Perspectives
- 根据标注者人口统计信息路由到不同专家网络，捕捉群体差异。
- 在高分歧数据集上表现优异，尤其提升弱势群体预测准确率。
- 结合大模型生成合成标注，有效补充真实数据不足。
我们提出一种通过架构与数据双重创新来建模主观自然语言处理任务中标注者分歧的方法。所提模型DEM-MoE（人口统计感知的专家混合模型）根据标注者的人口统计特征将输入路由至相应专家子网络，能更有效地表征群体层面的结构化差异。DEM-MoE在各人口统计群体中均表现稳健，在高标注分歧数据集上尤为突出。为应对人口统计覆盖稀疏问题，我们测试了利用大模型通过零样本角色提示生成合成标注的可行性。结果显示，这些合成判断与人类标注具有中等程度一致性，并可作为可扩展的数据扩充手段。进一步提出并评估了针对数据集结构定制的真-假数据融合策略，发现最优策略依赖于数据结构。整体贡献提升了对多元视角的建模能力。
原文摘要 · Abstract (English)
We present an approach to modeling annotator disagreement in subjective NLP tasks through both architectural and data-centric innovations. Our model, DEM-MoE (Demographic-Aware Mixture of Experts), routes inputs to expert subnetworks based on annotator demographics, enabling it to better represent structured, group-level variation compared to prior models. DEM-MoE consistently performs competitively across demographic groups, and shows especially strong results on datasets with high annotator disagreement. To address sparse demographic coverage, we test whether LLM-generated synthetic annotations via zero-shot persona prompting can be used for data imputation. We show these synthetic judgments align moderately well with human annotations on our data and offer a scalable way to potentially enrich training data. We then propose and evaluate approaches for blending real and synthetic data using strategies tailored to dataset structure. We find that the optimal strategies depend on dataset structure. Together, these contributions improve the representation of diverse perspectives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。