披露任务目标会诱导大模型生成偏见，影响金融预测可靠性。
Seeing the Goal, Missing the Truth: Human Accountability for AI Bias
- 通过暗示下游任务，大模型的中间输出会偏离初衷。
- 目标泄露使模型在截止前数据上表现更好，但无法提升截止后表现。
- 用户无意的对话提示也可能引发此类偏见，需警惕人为责任。
本研究探讨人类定义的目标如何通过目标条件认知影响大语言模型（LLM）的行为。以金融预测任务为例，我们发现：即使中间指标本应与下游任务无关，但一旦揭示其用途（如预测股价或盈利），大模型便会生成带有偏见的情感和竞争度量。目标感知提示会使中间指标向披露的目标倾斜，导致样本内过拟合。具体而言，目标泄露仅在大模型知识截止前的数据上提升性能，对截止后数据无帮助。这种偏见强烈到常规提示正则化也无法完全缓解。此外，偏见可能源于用户无意中透露的任务意图。总体而言，由‘看到目标’引发的AI偏见并非算法缺陷，而是研究设计中的人为责任问题。
原文摘要 · Abstract (English)
This research explores how human-defined goals influence the behavior of Large Language Models (LLMs) through purpose-conditioned cognition. Using financial prediction tasks, we show that revealing the downstream use (e.g., predicting stock returns or earnings) of LLM outputs leads the LLM to generate biased sentiment and competition measures, even though these measures are intended to be downstream task-independent. Goal-aware prompting shifts these intermediate measures toward the disclosed downstream objective, producing in-sample overfitting. Specifically, purpose leakage improves performance on data prior to the LLM's knowledge cutoff, but provides no advantage after the cutoff. This bias is strong enough that regularization of prompt instructions cannot fully address this form of overfitting. We further show that the bias can arise from users' unintentional conversational context that hints at the purpose. Overall, we document that AI bias due to "seeing the goal" is not an algorithmic flaw, but stems from human accountability in research design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。