研究大模型训练中的公平性动态，发现偏见会突然出现且不随性能指标变化。
Fairness Dynamics During Training
- 引入平均排名与分部JS散度新指标，全面评估训练过程中的偏见演化。
- Pythia-6.9b模型对男性职业预测更自信，准确率高于女性，存在明显性别偏见。
- 早停可使模型公平性提升92.5%，仅损失1.7%的准确性，适合需要低偏见场景。
我们研究大语言模型（LLM）训练过程中的公平性动态，以通过早期停止等训练干预手段诊断偏见并进行缓解；发现偏见可能突然出现，且并不总是与常见性能指标一致。为此，我们提出两个新指标：平均排名和分部詹森-香农散度，用于在预训练阶段整体评估公平性动态。这些指标揭示了Pythia模型在WinoBias数据集上的职业性别预测偏见演变过程：（1）Pythia-6.9b对男性偏向明显，训练过程中对“男性”预测更准确且更自信；（2）通过早停，该模型可在LAMBADA任务上牺牲1.7%准确率，换取92.5%的公平性提升；（3）更大模型可能表现出更强偏见，即使性别未指定，Pythia-6.9b也比Pythia-160m做出更多性别假设。
原文摘要 · Abstract (English)
We investigate fairness dynamics during Large Language Model (LLM) training to enable the diagnoses of biases and mitigations through training interventions like early stopping; we find that biases can emerge suddenly and do not always follow common performance metrics. We introduce two new metrics to evaluate fairness dynamics holistically during model pre-training: Average Rank and Jensen-Shannon Divergence by Parts. These metrics provide insights into the Pythia models' progression of biases in gender prediction of occupations on the WinoBias dataset. By monitoring these dynamics, we find that (1) Pythia-6.9b is biased towards men; it becomes more performant and confident predicting "male" than "female" during training, (2) via early-stopping, Pythia-6.9b can exchange 1.7% accuracy on LAMBADA for a 92.5% increase in fairness, and (3) larger models can exhibit more bias; Pythia-6.9b makes more assumptions about gender than Pythia-160m, even when a subject's gender is not specified.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。