arXiv:2604.16845cs.CL2026-04ACL被引 1

让大模型在识别群体差异时既准确又安全,避免因过度追求正确而产生有害解释。

DART: Mitigating Harm Drift in Difference-Aware LLMs via Distill-Audit-Repair Training

论文配图:DART: Mitigating Harm Drift in Difference-Aware LLMs via Distill-Audit-Repair Training
图 1 · 摘自论文原文
  • 通过蒸馏-审计-修复三阶段训练,动态检测并修正模型输出中的有害偏差。
  • 在8个基准测试中准确率从39.0%提升至68.8%,有害案例减少72.6%。
  • 适用于医疗、法律等现实场景,显著降低拒绝回答率至3.0%。

大语言模型在安全微调后常忽视人口差异,即使事实正确(如基于祖先的疾病发病率)或情境合理(如宗教招聘偏好)也不承认。这导致错误回应、无谓拒绝或泛化的“平等对待”默认。我们通过差异意识分类任务研究此问题:给定涉及群体的问题,不直接回答,而是判断正确回答是否需承认群体差异(是)或应同等对待(否)。关键发现:为提升准确率而微调,会引发有害性漂移——模型生成的解释在准确性提高时反而更危险,表现为夸大有害内容、引入问题假设或遗漏基线已识别的危害。为此,我们提出DART(Distill--Audit--Repair Training):从教师模型蒸馏标签条件推理,基于基线审计输出中的危害漂移,通过严重度加权微调修复问题。在八个基准上,DART将Llama-3-8B-Instruct准确率从39.0%提升至68.8%,在同等对待提示上的提升最大(11.3%→72.6%),同时将危害漂移案例减少72.6%。该方法还可迁移至280个开放式真实世界查询(涵盖医疗、法律、政策、教育领域),使适当区分群体的回答比例从39.8%提升至77.5%,拒绝率从34.3%降至3.0%。结果表明,只要具备显式检测与修复机制,准确性和安全性可兼得。

原文摘要 · Abstract (English)

Large language models (LLMs) tuned for safety often avoid acknowledging demographic differences, even when such acknowledgment is factually correct (e.g., ancestry-based disease incidence) or contextually justified (e.g., religious hiring preferences). This identity-blindness yields incorrect responses, unnecessary refusals, or generic "equal-treatment" defaults. We study this via difference-awareness classification: given a question involving demographic groups, the task is not to answer directly, but to classify whether a correct answer requires recognizing group differences (yes) or whether groups should be treated identically (no). Crucially, fine-tuning for accuracy triggers harm drift: model-generated explanations become increasingly harmful as decision accuracy improves, whether by elaborating harmful content, introducing problematic assumptions, or failing to flag harms the baseline identified. To mitigate this, we introduce DART (Distill--Audit--Repair Training), which distills label-conditioned reasoning from a teacher, audits outputs for harm drift cases relative to baseline, and repairs problematic cases via severity-weighted fine-tuning. On eight benchmarks, DART improves Llama-3-8B-Instruct accuracy from 39.0% to 68.8%, with largest gains on equal-treatment prompts (11.3% -> 72.6%), while reducing harm drift cases by 72.6%. It also transfers to 280 open-ended real-world queries across medical, legal, policy, and educational domains, improving difference-appropriate responses from 39.8% to 77.5% while reducing refusals from 34.3% to 3.0%. Our results demonstrate that accuracy and safety need not conflict when explicit detection and repair mechanisms are in place.

大模型安全偏见缓解差异常识微调方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。