arXiv:2608.22820cs.LGcs.AI2026-08

通过重加权与门控正则化,让专家模型更公平地服务不同人群。

Fairness-Aware Mixture-of-Experts via Subgroup Reweighting and Gate Regularization

论文配图:Fairness-Aware Mixture-of-Experts via Subgroup Reweighting and Gate Regularization
图 1 · 摘自论文原文
  • 用子群体重加权缓解数据不平衡,门控熵正则防止路由偏斜。
  • 在多个数据集上显著提升公平性指标,预测性能不降反升。
  • 适合关注模型公平性且需要可解释路由的AI应用开发者。

深度学习模型常因性别、年龄等敏感属性的数据分布不均,导致不同人口群体间性能差异。现有方法多分为单阶段和两阶段:前者共享表示但难处理异质子群体分布,后者虽分别学习表示却易与下游任务目标错配。本文识别出一种新问题——路由诱导偏差,即子群体不平衡使门控网络将多数子群体集中分配至少数专家。为此提出端到端的Mixture-of-Experts(MoE)框架,结合子群体重加权以修正数据不平衡,并引入门控熵正则化防止路由坍缩至子群体属性,确保专家使用均衡且可解释。实验表明,该方法在保持优异预测性能的同时显著提升公平性,同时路由分布提供了子群体分配的可解释视图。

原文摘要 · Abstract (English)

Deep learning models often produce performance disparities across demographic groups, due to the training data imbalance with respect to sensitive attributes such as gender or age. To address this problem, existing work has explored fair representation learning, data re-sampling, and adversarial training, which can be broadly categorized into two main approaches. Single-stage methods typically learn a shared representation for fairness, but often struggle to handle heterogeneous subgroup distributions. Two-stage methods learn representations separately from the final prediction task, which can lead to misalignment between fairness objectives and downstream predictions. We identify routing-induced bias, a failure mode in which subgroup imbalance drives the gating network to route subgroups onto a few experts, and propose an end-to-end Mixture-of-Experts (MoE) framework that corrects it. Specifically, we apply subgroup reweighting to correct data imbalance, and introduce gate entropy regularization to prevent routing from collapsing onto subgroup attributes, keeping expert utilization both balanced and interpretable. Beyond improving fairness, the routing distribution offers an interpretable view of how subgroups are allocated across experts. Experimental results demonstrate that the proposed approach improves fairness while maintaining competitive predictive performance.

公平性MoE门控机制可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。