用大模型自动标注政治新闻真伪，提升事实核查效率与透明度。
Fact or Fiction? Can LLMs be Reliable Annotators for Political Truths?
- 用开源大模型生成政治新闻的真假标注数据
- 人工专家与大模型双验证，确保标注可靠性
- 适合需要大规模事实核查的媒体机构与研究者
政治虚假信息对民主进程构成重大挑战，影响公众意见与媒体信任。传统人工核查存在可扩展性差和标注者偏见问题，而机器学习模型又依赖昂贵的大规模标注数据。本研究探讨了当前最先进的大语言模型（LLMs）作为政治新闻事实性检测标注者的可行性。我们利用开源大模型构建了一个政治多样性数据集，并通过大模型生成的标注来标记偏见。这些标注经由人类专家验证，并进一步由大模型裁判评估，以检验标注的准确性和可靠性。该方法为传统事实核查提供了一种可扩展且稳健的替代方案，有助于增强媒体透明度与公众信任。
原文摘要 · Abstract (English)
Political misinformation poses significant challenges to democratic processes, shaping public opinion and trust in media. Manual fact-checking methods face issues of scalability and annotator bias, while machine learning models require large, costly labelled datasets. This study investigates the use of state-of-the-art large language models (LLMs) as reliable annotators for detecting political factuality in news articles. Using open-source LLMs, we create a politically diverse dataset, labelled for bias through LLM-generated annotations. These annotations are validated by human experts and further evaluated by LLM-based judges to assess the accuracy and reliability of the annotations. Our approach offers a scalable and robust alternative to traditional fact-checking, enhancing transparency and public trust in media.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。