arXiv:2602.18351cs.CLcs.AI2026-02被引 1

提出双尺度验证框架,提升政治立场预测的可靠性。

Validating Political Position Predictions of Arguments

  • 结合点对点与成对标注,平衡评估可扩展性与准确性。
  • 最佳模型在排序一致性上达α=0.86,显著高于点对点一致率(α=0.578)。
  • 适合从事政治话语分析、知识图谱构建的研究者使用。

现实世界知识表示常需捕捉主观且连续的属性(如政治立场),但这类属性与广泛接受的成对验证标准存在冲突。本文通过双尺度验证框架,应用于论辩话语中的政治立场预测,融合点对点与成对人工标注。利用22个语言模型,构建了涵盖30场英国电视节目《问答时间》辩论中23,228条论据的政治立场预测大规模知识库。点对点评估显示人类与模型间中等一致性(Krippendorff's α=0.578),反映内在主观性;而成对验证揭示人类与模型排名间显著更强的一致性(最佳模型α=0.86)。本工作贡献:(i) 一种兼顾可扩展性与可靠性的主观连续知识验证方法;(ii) 一个经验证的结构化论辩知识库,支持图推理与检索增强生成;(iii) 证明从本质上主观的现实话语中可提取有序结构,推动传统符号或分类方法不足领域的知识表示能力。

原文摘要 · Abstract (English)

Real-world knowledge representation often requires capturing subjective, continuous attributes -- such as political positions -- that conflict with pairwise validation, the widely accepted gold standard for human evaluation. We address this challenge through a dual-scale validation framework applied to political stance prediction in argumentative discourse, combining pointwise and pairwise human annotation. Using 22 language models, we construct a large-scale knowledge base of political position predictions for 23,228 arguments drawn from 30 debates that appeared on the UK politicial television programme \textit{Question Time}. Pointwise evaluation shows moderate human-model agreement (Krippendorff's $α=0.578$), reflecting intrinsic subjectivity, while pairwise validation reveals substantially stronger alignment between human- and model-derived rankings ($α=0.86$ for the best model). This work contributes: (i) a practical validation methodology for subjective continuous knowledge that balances scalability with reliability; (ii) a validated structured argumentation knowledge base enabling graph-based reasoning and retrieval-augmented generation in political domains; and (iii) evidence that ordinal structure can be extracted from pointwise language models predictions from inherently subjective real-world discourse, advancing knowledge representation capabilities for domains where traditional symbolic or categorical approaches are insufficient.

政治立场知识表示双尺度验证论辩分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。