arXiv:2603.19144cs.CLcs.AI2026-03

通过图结构建模模型内部表示,实现对大模型偏见的深层修正。

UGID: Unified Graph Isomorphism for Debiasing Large Language Models

  • 将Transformer视为带注意力路由的结构化图,从内部表征层面干预偏见
  • 在分布内与分布外场景下均显著降低偏见,且保持模型安全与通用能力
  • 适合关注模型公平性、可解释性的研究人员与开发者

大语言模型存在显著的社会偏见。现有基于输出或数据优化的去偏方法无法彻底解决此问题,多项研究已表明偏见嵌入于内部表征中。本文提出统一图同构去偏框架UGID,将Transformer建模为结构化计算图:注意力机制定义图的边,隐藏状态定义节点。去偏被形式化为在反事实输入下保持图结构不变,仅允许敏感属性变化。UGID联合约束注意力路由与偏见敏感区域的隐藏表示,有效防止偏见在架构组件间迁移。为在不损害通用能力的前提下实现行为对齐,引入敏感逻辑值的对数空间约束及选择性锚点目标以保留定义语义。大量实验表明,UGID在分布内与分布外设置下均能有效降低偏见,显著减少内部结构差异,并维持模型安全性与实用性。

原文摘要 · Abstract (English)

Large language models (LLMs) exhibit pronounced social biases. Output-level or data-optimization--based debiasing methods cannot fully resolve these biases, and many prior works have shown that biases are embedded in internal representations. We propose \underline{U}nified \underline{G}raph \underline{I}somorphism for \underline{D}ebiasing large language models (\textit{\textbf{UGID}}), an internal-representation--level debiasing framework for large language models that models the Transformer as a structured computational graph, where attention mechanisms define the routing edges of the graph and hidden states define the graph nodes. Specifically, debiasing is formulated as enforcing invariance of the graph structure across counterfactual inputs, with differences allowed only on sensitive attributes. \textit{\textbf{UGID}} jointly constrains attention routing and hidden representations in bias-sensitive regions, effectively preventing bias migration across architectural components. To achieve effective behavioral alignment without degrading general capabilities, we introduce a log-space constraint on sensitive logits and a selective anchor-based objective to preserve definitional semantics. Extensive experiments on large language models demonstrate that \textit{\textbf{UGID}} effectively reduces bias under both in-distribution and out-of-distribution settings, significantly reduces internal structural discrepancies, and preserves model safety and utility.

大模型去偏图神经网络公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。