用用户历史发帖预测立场，大模型表现可行但波动大。
Predicting User Stances from Target-Agnostic Information using Large Language Models
- 基于用户无目标发帖，用大模型推断其立场
- 不同话题/策略/帖子数下效果差异明显
- 关键词和价值观特征对预测有帮助
我们研究大语言模型（LLMs）在仅凭用户历史无目标社交帖子的情况下，预测其对某一议题立场的能力（即用户级立场预测）。尽管初步证据表明大模型具备此能力，但我们发现模型表现存在显著差异，受议题类型、预测策略以及提供帖子数量的影响。事后分析进一步表明，这些无目标帖子通过表面关键词（如与议题相关词汇）和用户层面特征（如编码用户道德观）为大模型提供了有用信息。总体而言，我们的研究提示大模型或可成为基于历史无目标数据推断公众对新议题立场的可行方法。同时，我们也呼吁进一步研究以深入理解大模型在该任务上的强表现及其在不同情境下的有效性变化。
原文摘要 · Abstract (English)
We investigate Large Language Models' (LLMs) ability to predict a user's stance on a target given a collection of his/her target-agnostic social media posts (i.e., user-level stance prediction). While we show early evidence that LLMs are capable of this task, we highlight considerable variability in the performance of the model across (i) the type of stance target, (ii) the prediction strategy and (iii) the number of target-agnostic posts supplied. Post-hoc analyses further hint at the usefulness of target-agnostic posts in providing relevant information to LLMs through the presence of both surface-level (e.g., target-relevant keywords) and user-level features (e.g., encoding users' moral values). Overall, our findings suggest that LLMs might offer a viable method for determining public stances towards new topics based on historical and target-agnostic data. At the same time, we also call for further research to better understand LLMs' strong performance on the stance prediction task and how their effectiveness varies across task contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。