不用微调,直接用提示词提取文本情感,效果比传统方法更好
This Candidate is [MASK]. Prompt-based Sentiment Extraction and Reference Letters
- 用提示词直接提取情感分数,无需预处理或标注数据
- 参考信情感越高,求职者表现越好,情感波动大则影响更大
- 可区分性别倾向,发现女性信件更强调勤奋特质
本文提出一种基于提示词的情感提取方法,无需微调或标注数据,直接对原始文本输入生成具有概率意义的情感分数。该方法在经济学与金融学领域中具有明显优势。作者将其应用于手工收集的机密推荐信(Reference Letters, RLs)语料库,发现推荐信中的情感内容显著影响求职结果:平均情感值更高的候选人,在各类成功指标下表现均更优;而推荐人之间情感差异越大,候选人的求职表现越差。相比词袋模型、微调语言模型和调用高级聊天机器人等方法,该提示词策略能唯一复现上述结论。此外,通过微调方法获得性别化情感分数,发现针对女性候选人的推荐信更强调‘勤勉’特质,而男性则突出‘出众’特质,这种性别偏见对女性求职结果产生负面影响。
原文摘要 · Abstract (English)
I propose a relatively simple way to deploy pre-trained large language models (LLMs) in order to extract sentiment and other useful features from text data. The method, which I refer to as prompt-based sentiment extraction, offers multiple advantages over other methods used in economics and finance. In particular, it accepts the text input as is (without pre-processing) and produces a sentiment score that has a probability interpretation. Unlike other LLM-based approaches, it does not require any fine-tuning or labeled data. I apply my prompt-based strategy to a hand-collected corpus of confidential reference letters (RLs). I show that the sentiment contents of RLs are clearly reflected in job market outcomes. Candidates with higher average sentiment in their RLs perform markedly better regardless of the measure of success chosen. Moreover, I show that sentiment dispersion among letter writers negatively affects the job market candidate's performance. I compare my sentiment extraction approach to other commonly used methods for sentiment analysis: `bag-of-words' approaches, fine-tuned language models, and querying advanced chatbots. No other method can fully reproduce the results obtained by prompt-based sentiment extraction. Finally, I slightly modify the method to obtain `gendered' sentiment scores (as in Eberhardt et al., 2023). I show that RLs written for female candidates emphasize `grindstone' personality traits, whereas male candidates' letters emphasize `standout' traits. These gender differences negatively affect women's job market outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。