用Transformer模型量化德国文本的政治倾向,从左到右连续打分。
Ideology Prediction of German Political Texts

- 构建四个语料库,用DeBERTa-large等模型预测文本政治倾向。
- 在德国议会记录上达到0.844的F1分数,推特数据上准确率达86.4%。
- 强调领域数据比模型大小更重要,适合政治分析与舆情研究者。
选举是国家发展的重要节点。为更好理解从左到右各政治运动的修辞,我们提出一种基于Transformer的模型,可将文本政治倾向映射为-1到1之间的连续标量d。该方法使分析者能聚焦特定政治光谱区间(如保守派),排除其他阵营。传统多分类器受限于预定义类别,而本方法突破此限制。为筛选最优基础模型,我们构建了四个语料库:德国联邦议院全体会议记录标注集、官方决策工具Wahl-O-Mat数据、33家具政治倾向标识的报纸文章,以及597名议员的53.52万条推文。通过使用两个不同语料库分别训练与测试以缓解过拟合。在域内表现中,DeBERTa-large取得最高F1分数0.844;在域外推特测试中,准确率达86.4%;报纸数据上,Gemma2-2B表现最佳,平均绝对误差为0.172。结果表明,变压器模型可达到公众舆论调查水平的政治语境识别能力。研究指出,模型架构与领域数据可用性对偏见估计的影响可能与模型规模同等重要。我们讨论了方法局限性,并提出了提升偏见测量鲁棒性的方向。
原文摘要 · Abstract (English)
Elections represent a crucial milestone in a nation's ongoing development. To better understand the political rhetoric from various movements, ranging from left to right, we propose a transformer-based model capable of projecting the political orientation of a text on a continuous left-to-right spectrum, represented by a normalized scalar d between -1 and 1. This approach enables analysts to focus on specific segments of the political landscape, such as conservatives, while excluding liberal and far-right movements. Such a task can only be achieved with multiclass classifiers, provided that the desired orientation is incorporated within one of their predefined classes. To determine the most suitable foundation model among 13 candidate transformers for this task, we constructed four distinct corpora. One corpus comprised annotated plenary notes from the German Bundestag, while another was based on an official online decision-making tool, Wahl-O-Mat. The third corpus consisted of articles from 33 newspapers, each identified by its political orientation, and the fourth included 535,200 tweets from 597 members of the 20th and 21st German Bundestag. To mitigate overfitting, we used two distinct corpora for training and two for testing, respectively. For in-domain performance, DeBERTa-large achieved the highest F1 score F1=0.844 as well as for the X (Twitter) out-of-domain test ACC=0.864. Regarding the newspaper out-of-domain test, Gemma2-2B excelled (MAE = 0.172). This study demonstrates that transformer models can recognize political framing in German news at the level of public opinion polls. Our findings suggest that both the model architecture and the availability of domain-specific training data can be as influential as model size for estimating political bias. We discuss methodological limitations and outline directions for improving the robustness of bias measurement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。