动态验证选例提升大模型公平性，避免偏见输出。
SMITE: Enhancing Fairness in LLMs through Optimal In-Context Example Selection via Dynamic Validation
- 用动态验证集替代静态验证，实时调整示例选择标准。
- 在4个大模型上测试，准确率与公平性均显著优于基线。
- 适合关注模型公平性与可解释性的研究者与工程师。
大语言模型（LLMs）广泛应用于表格分类等下游任务，确保其输出公平性对实现包容性、平等代表及负责任的AI部署至关重要。本文提出一种新方法，通过动态验证集（随测试集演化）替代传统静态验证，构建迭代算法SMITE以选择最优上下文示例。每个示例集均在对应动态验证集上进行评估，选取总误差最低的作为最终演示集。在四个不同大模型上的实验表明，该方法显著提升了预测准确率与公平性。据我们所知，这是首个将动态验证引入大模型上下文学习的研究。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are widely used for downstream tasks such as tabular classification, where ensuring fairness in their outputs is critical for inclusivity, equal representation, and responsible AI deployment. This study introduces a novel approach to enhancing LLM performance and fairness through the concept of a dynamic validation set, which evolves alongside the test set, replacing the traditional static validation approach. We also propose an iterative algorithm, SMITE, to select optimal in-context examples, with each example set validated against its corresponding dynamic validation set. The in-context set with the lowest total error is used as the final demonstration set. Our experiments across four different LLMs show that our proposed techniques significantly improve both predictive accuracy and fairness compared to baseline methods. To our knowledge, this is the first study to apply dynamic validation in the context of in-context learning for LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。