用大模型更准确地分解性别工资差距,发现传统方法遗漏了关键职业因素。
Estimating Wage Disparities Using Foundation Models
- 通过改进微调策略,让大模型在估计时避免遗漏变量偏差。
- 在收入动态面板数据中发现,职业经历解释的性别工资差距比传统方法高23%。
- 适合做劳动力市场分析、社会不平等研究的研究者参考。
基础模型的兴起标志着机器学习范式的转变:不再从头训练专用模型,而是先在大规模数据上预训练基础模型,再针对小规模数据进行适配或微调。虽然基础模型最初用于文本,但也在社会科学研究预测任务中表现出色。然而,社会科学中的许多估计问题虽以预测为中间步骤,却需不同的成功标准。本文提出针对此类估计问题的微调方法。我们首先揭示仅以预测准确性为目标微调时可能产生的遗漏变量偏差。随后,给出一组新条件,确保基于基础模型的估计量达到根n一致性。基于此理论,我们设计了能有效缓解该偏差的新微调算法。为验证方法,我们研究了性别工资分解问题——即在计量经济学中将性别工资差距分解为可由职业经历解释与不可解释的部分。经典方法依赖简化的工资预测模型,仅使用职业经历的粗略摘要,可能遗漏重要变量。我们采用自建基础模型,捕捉更丰富的职业经历信息。利用收入动态面板数据(Panel Study of Income Dynamics),结果表明,职业经历解释的性别工资差距超过传统模型测量值的23%,并识别出被传统模型忽略但对解释差距至关重要的职业特征。
原文摘要 · Abstract (English)
The rise of foundation models marks a paradigm shift in machine learning: instead of training specialized models from scratch, foundation models are first trained on massive datasets before being adapted or fine-tuned to make predictions on smaller datasets. Initially developed for text, foundation models have also excelled at making predictions about social science data. However, while many estimation problems in the social sciences use prediction as an intermediate step, they ultimately require different criteria for success. In this paper, we develop methods for fine-tuning foundation models to perform these estimation problems. We first characterize an omitted variable bias that can arise when a foundation model is only fine-tuned to maximize predictive accuracy. We then provide a novel set of conditions for fine-tuning under which estimates derived from a foundation model are root-n-consistent. Based on this theory, we develop new fine-tuning algorithms that empirically mitigate this omitted variable bias. To demonstrate our ideas, we study gender wage decomposition. This is a statistical estimation problem from econometrics where the goal is to decompose the gender wage gap into components that can and cannot be explained by career histories of workers. Classical methods for decomposing the wage gap employ simple predictive models of wages which condition on coarse summaries of career history that may omit factors that are important for explaining the gap. Instead, we use a custom-built foundation model to decompose the gender wage gap, which captures a richer representation of career history. Using data from the Panel Study of Income Dynamics, we find that career history explains more of the gender wage gap than standard econometric models can measure, and we identify elements of career history that are omitted by standard models but are important for explaining the wage gap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。