用NLP分析总统指令中的策略信号，发现工具有效但仍有偏差。
Assessing the Applicability of Natural Language Processing to Traditional Social Science Methodology: A Case Study in Identifying Strategic Signaling Patterns in Presidential Directives
- 用NLP从总统指令中提取主题，对比人工标注结果。
- 两种方法识别出相似信号主题，但存在不一致之处。
- 适合关注AI在社科研究中应用的学者参考。
本研究探讨自然语言处理（NLP）在社会科学方法中的适用性，以里根至克林顿执政期间的总统指令（PDs）为案例，分析其策略性信号模式。通过对比人工与NLP识别的结果，验证了在大规模文本分析中使用NLP的潜力。尽管两者均能识别出相关文档和主题，但仍存在差异，表明当前NLP在该场景下的有效性仍需进一步评估。研究于2023年完成，反映了当时较陈旧的AI工具在新兴社会科学研究中的实际表现，提示现有技术虽有进步，但尚不足以完全替代人类判断。
原文摘要 · Abstract (English)
Our research investigates how Natural Language Processing (NLP) can be used to extract main topics from a larger corpus of written data, as applied to the case of identifying signaling themes in Presidential Directives (PDs) from the Reagan through Clinton administrations. Analysts and NLP both identified relevant documents, demonstrating the potential utility of NLPs in research involving large written corpuses. However, we also identified discrepancies between NLP and human-labeled results that indicate a need for more research to assess the validity of NLP in this use case. The research was conducted in 2023, and the rapidly evolving landscape of AIML means existing tools have improved and new tools have been developed; this research displays the inherent capabilities of a potentially dated AI tool in emerging social science applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。