用大模型上下文学习填补民意调查缺失数据，效果优于传统方法。
In-Context Learning for the Imputation of Public Opinion Data with Large Language Models
- 利用大模型上下文学习机制填补调查中部分缺失值
- 在非随机缺失场景下误差降低，置信区间窄2至5倍
- 适合需要高精度民意推断的研究者与政策分析人员
大语言模型已被广泛用于模拟个体调查回答。然而实际中，完全未观测的回答很少；主要问题是部分缺失。插补旨在通过填补缺失值恢复调查数据的整体结构，具有独立的评估标准，与预测有本质区别。本文提出通过上下文学习（ICL）进行缺失调查数据的插补。我们在15个波次、共150个民意变量的美国趋势面板数据上，系统评估了不同缺失机制（MCAR、MAR、MNAR）下的ICL设计选择。结果表明，相比成熟的统计插补方法如MICE PMM，ICL方法在所有缺失机制下均显著降低绝对误差，尤其在非随机缺失（MNAR）时提升最明显。最佳配置（gpt-oss-120b + 100个上下文示例）实现接近名义上的总体覆盖率（接近95%），置信区间宽度仅为MICE PMM的二至五分之一。我们发布了配套Python包，提供类似sklearn的API，支持本地及私有大模型部署。
原文摘要 · Abstract (English)
Large language models have been widely evaluated as simulators of individual survey responses. In practice, however, fully unobserved responses are rare; the dominant problem is partial non-response. Imputation aims to restore the overall structure of a survey dataset by filling in these missing values. It has its own well-defined evaluation criteria and differs fundamentally from prediction. We propose to impute missing survey data through in-context learning (ICL). We systematically evaluate ICL design choices across different missingness mechanisms (MCAR, MAR, MNAR) on 150 opinion variables spanning 15 waves of the American Trends Panel. Compared to well-established statistical methods for data imputation like MICE PMM, our ICL approach consistently reduces absolute error across all missingness mechanisms, with the largest gains under non-random missingness (MNAR). Notably, the best-performing specification (gpt-oss-120b with 100 in-context examples) achieves near-nominal aggregate coverage (approaching the 95% level) with confidence intervals two to five times narrower than MICE PMM. We publish a Python package with an sklearn-like API to enable easy deployment of our method using local and proprietary LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。