用网址判断新闻政治性，高效但易误判中间派内容。
Beyond the Link: Assessing LLMs' ability to Classify Political Content across Global Media
- 通过分析网页链接和文本，评估大模型跨国家识别政治内容的能力。
- 网址含有效信息，可替代全文分析，节省成本且保持较高准确率。
- 模型对中间派新闻误判率高，可能影响后续研究结果的可靠性。
大型语言模型（LLMs）在政治科学与数字媒体研究中的应用日益广泛。尽管其在标注任务中表现良好，但利用网页链接（URL）对政治内容（PC）进行分类的有效性仍缺乏充分探索。本文评估了大模型在五个国家（法国、德国、西班牙、英国和美国）不同语言的新闻文章中，仅基于文本和链接判断政治内容的能力。采用前沿模型，对比人工标注数据，检验链接层面分析能否近似全文字分析。结果显示，链接包含相关线索，可作为可扩展、低成本的替代方案以识别政治内容。然而，研究也发现系统性偏差：模型倾向于将中间派新闻误判为政治内容，导致假阳性，可能扭曲后续分析。文章最后提出在政治科学研究中使用大模型的方法论建议。
原文摘要 · Abstract (English)
The use of large language models (LLMs) is becoming common in political science and digital media research. While LLMs have demonstrated ability in labelling tasks, their effectiveness to classify Political Content (PC) from URLs remains underexplored. This article evaluates whether LLMs can accurately distinguish PC from non-PC using both the text and the URLs of news articles across five countries (France, Germany, Spain, the UK, and the US) and their different languages. Using cutting-edge models, we benchmark their performance against human-coded data to assess whether URL-level analysis can approximate full-text analysis. Our findings show that URLs embed relevant information and can serve as a scalable, cost-effective alternative to discern PC. However, we also uncover systematic biases: LLMs seem to overclassify centrist news as political, leading to false positives that may distort further analyses. We conclude by outlining methodological recommendations on the use of LLMs in political science research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。