用大模型分析文本因果，判断治疗是否影响文本内容及影响范围。
Causal Inference on Outcomes Learned from Text
- 用大模型识别两组文本的系统性差异，定位处理效应
- 通过样本分割和人工标注验证，确保结论可靠
- 适合医学、社会科学等需从文本中挖掘因果效应的研究
我们提出一种机器学习工具，用于在随机试验中对文本所蕴含的结果进行因果推断。基于一个简单的计量经济学框架，该方法回答三个问题：文本是否受处理影响？处理效应作用于哪些结果？因果效应描述的完整性如何？为此，方法利用大语言模型（LLMs）识别两组文本文档间的系统性差异，并基于高成本验证提供有效推断。具体包括：强调样本分割以实现对LLM输出的统计验证，以及依赖人工标注验证文档间实质性差异。我们在学术论文摘要的原型应用中展示了该工具的有效性。
原文摘要 · Abstract (English)
We propose a machine-learning tool that yields causal inference on text in randomized trials. Based on a simple econometric framework in which text may capture outcomes of interest, our procedure addresses three questions: First, is the text affected by the treatment? Second, which outcomes is the effect on? And third, how complete is our description of causal effects? To answer all three questions, our approach uses large language models (LLMs) that suggest systematic differences across two groups of text documents and then provides valid inference based on costly validation. Specifically, we highlight the need for sample splitting to allow for statistical validation of LLM outputs, as well as the need for human labeling to validate substantive claims about how documents differ across groups. We illustrate the tool in a proof-of-concept application using abstracts of academic manuscripts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。