用大模型隐状态预测经济数据，效果比直接输出更好。
Revealing economic facts: LLMs know more than they say
- 用大模型隐状态做线性回归，比文本输出更准
- 几十个标注样本就能训练出高精度模型
- 无需目标变量标签即可迁移提升预测效果
我们研究大型语言模型（LLMs）的隐藏状态是否可用于估算和填补经济与金融统计数据。聚焦于县级（如失业率）和企业级（如总资产）变量，发现仅需在开源大模型的隐藏状态上训练简单线性模型，其表现就优于模型的文本输出。这表明隐藏状态蕴含的经济信息比直接输出更丰富。学习曲线分析显示，仅需数十个标注样本即可完成有效训练。我们还提出一种迁移学习方法，在不依赖目标变量标注数据的情况下提升了估计精度。最后，我们在超分辨率和数据填补任务中展示了隐藏状态表示的实际应用价值。
原文摘要 · Abstract (English)
We investigate whether the hidden states of large language models (LLMs) can be used to estimate and impute economic and financial statistics. Focusing on county-level (e.g. unemployment) and firm-level (e.g. total assets) variables, we show that a simple linear model trained on the hidden states of open-source LLMs outperforms the models' text outputs. This suggests that hidden states capture richer economic information than the responses of the LLMs reveal directly. A learning curve analysis indicates that only a few dozen labelled examples are sufficient for training. We also propose a transfer learning method that improves estimation accuracy without requiring any labelled data for the target variable. Finally, we demonstrate the practical utility of hidden-state representations in super-resolution and data imputation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。