用开源模型+结构化推理,让新闻框架分析更透明可审计。
Framing Migration News with LLMs: Structured CoT as a Support for Human Interpretation

- 用Llama3-8B加结构化思维链,分步解释新闻框架类别
- 在单张显卡上运行,准确率高于零样本和少样本基线
- 帮助研究者看清模型逻辑,但可能悄悄影响人的判断
迁移新闻的框架分析是一项具有社会意义的任务:媒体学者与研究人员需要既准确又透明、可审计且在典型学术研究组资源限制下可访问的工具。现有基于大模型的方法依赖专有API和大型模型,引发数据隐私、可复现性及获取公平性的担忧。本文研究本地部署的开源大模型如何作为辅助工具支持可解释的框架分析。我们提出一种结构化思维链(SCoT)提示方法,使用Llama3-8B,通过预定义的框架类别进行分步推理,使输出可审计并支持对不同解释的检视。在迁移相关新闻数据集上的评估显示,SCoT在分类性能上优于零样本和少样本基线,且可在单张GPU上实现。人类中心评估中,标注者评价模型推理的连贯性和影响力:平均得分4.1/5,虽存在文本间差异,但多数认为推理合理,并能引发对初始判断的反思,即使存在分歧。研究揭示了大模型辅助框架分析的潜力与风险——结构化推理提升了输出可追溯性,支持批判性解读,但也可能以微妙方式影响人类判断。通过支持本地部署与强调人机协作,本工作推动了面向社会影响力媒体叙事的负责任、可及计算工具讨论。
原文摘要 · Abstract (English)
Frame analysis of migration news is a socially consequential task: media scholars and researchers who study how migration is narrated need tools that are not only accurate, but transparent, auditable, and accessible within the resource constraints typical of academic research groups. Existing LLM-based approaches rely on proprietary APIs and large models that raise concerns about data privacy, reproducibility and equitable access among media researchers. This work studies how a locally deployable open-source LLM can support interpretable frame analysis as an assistive tool. We introduce a Structured Chain-of-Thought (SCoT) prompting approach using Llama3-8B, enabling step-by-step justifications grounded in predefined framing categories. This structured design allows users to audit model outputs and examine alternative interpretations in a task that is inherently subjective. We evaluate our approach on a dataset of migration-related news and show that SCoT improves classification performance over zero-shot and few-shot baselines while remaining feasible on a single GPU. Then, we conduct a human-centered evaluation in which annotators assess the coherence and influence of "the model's reasoning". Results indicate that SCoT explanations are generally perceived as logical (mean score 4.1/5, though with notable variation across texts) and can prompt reflection on initial interpretations, even when disagreement persists. Our findings highlight both the potential and risks of LLM-assisted frame analysis. While structured reasoning can increase the traceability of model outputs and support critical interpretation, it can also influence human judgment in subtle ways. By enabling local deployment and emphasizing human-in-the-loop interaction, this work contributes to discussions on responsible and accessible computational tools for the study of socially impactful media narratives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。