让相似医学题答得一样，提升大模型答题公平性与准确率
Metric-Fair Prompting: Treating Similar Samples Similarly
- 基于语义相似度成对处理题目，强制相似问题输出一致
- 引入类利普希茨约束，确保相似输入得分相近，输出一致
- 适用于医疗等高风险场景，适合关注模型公平性的研究者
我们提出「度量公平提示」（Metric-Fair Prompting），一种面向公平性的提示框架，引导大语言模型在度量公平性约束下做出决策。在多项选择医学问答任务中，每个{(问题,选项)}对被视为二分类样本，标签为+1（正确）或-1（错误）。为实现个体公平——即对待相似样本一致——我们使用自然语言嵌入计算问题相似度，并对相似问题的联合配对进行求解，而非孤立处理。提示策略强制执行全局决策协议：提取决定性临床特征，将每个(问题,选项)映射为一个分数f(x)，作为置信度，并施加类利普希茨约束，使相似输入获得相似分数，从而保证输出一致性。在MedQA（US）基准上的评估显示，该方法优于标准单题提示，表明基于公平性、以置信度为导向的推理能提升大模型在高风险临床多选题中的准确性。
原文摘要 · Abstract (English)
We introduce \emph{Metric-Fair Prompting}, a fairness-aware prompting framework that guides large language models (LLMs) to make decisions under metric-fairness constraints. In the application of multiple-choice medical question answering, each {(question, option)} pair is treated as a binary instance with label $+1$ (correct) or $-1$ (incorrect). To promote {individual fairness}~--~treating similar instances similarly~--~we compute question similarity using NLP embeddings and solve items in \emph{joint pairs of similar questions} rather than in isolation. The prompt enforces a global decision protocol: extract decisive clinical features, map each \((\text{question}, \text{option})\) to a score $f(x)$ that acts as confidence, and impose a Lipschitz-style constraint so that similar inputs receive similar scores and, hence, consistent outputs. Evaluated on the {MedQA (US)} benchmark, Metric-Fair Prompting is shown to improve performance over standard single-item prompting, demonstrating that fairness-guided, confidence-oriented reasoning can enhance LLM accuracy on high-stakes clinical multiple-choice questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。