识别观点类论证中的主观性，让标注更精准。
Will Annotators Disagree? Identifying Subjectivity in Value-Laden Arguments
- 直接识别主观性比预测价值更有效
- 结合对比损失与交叉熵可降低对标签依赖
- 适合需要理解多元视角的标注任务
将多份标注合并为单一真实标签可能掩盖标注者分歧，尤其在涉及主观判断的任务中。本文研究如何识别论证中体现人类价值观的主观性，比较了通过价值预测推断主观性与直接识别主观性两种方法。实验表明，直接识别主观性显著提升模型标记主观论证的性能。此外，结合对比损失与二元交叉熵损失虽未提升性能，但降低了对单标签主观性的依赖。所提方法有助于发现个体可能不同解读的论证，推动更细致的标注流程。
原文摘要 · Abstract (English)
Aggregating multiple annotations into a single ground truth label may hide valuable insights into annotator disagreement, particularly in tasks where subjectivity plays a crucial role. In this work, we explore methods for identifying subjectivity in recognizing the human values that motivate arguments. We evaluate two main approaches: inferring subjectivity through value prediction vs. directly identifying subjectivity. Our experiments show that direct subjectivity identification significantly improves the model performance of flagging subjective arguments. Furthermore, combining contrastive loss with binary cross-entropy loss does not improve performance but reduces the dependency on per-label subjectivity. Our proposed methods can help identify arguments that individuals may interpret differently, fostering a more nuanced annotation process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。