arXiv:2608.00640cs.CL2026-08

首个针对藏医文化偏见的评测基准,揭示大模型在藏医知识上的系统性偏差。

TreeProbe : A Tibetan Medicine Benchmark for Cultural Bias in LLMs

论文配图:TreeProbe : A Tibetan Medicine Benchmark for Cultural Bias in LLMs
图 1 · 摘自论文原文
  • 基于藏医本体树框架构建多任务评测集
  • 4719个专家标注样本显示模型普遍表现不佳
  • 揭示模型因预训练数据差异向西医或中医偏移

大型语言模型被寄望于缓解全球健康不平等,但其输出常反映主流高资源医学传统,对传统医学知识覆盖不足。藏医是世界四大传统医学体系之一,具有独立且结构化的理论体系。当模型缺乏对藏医的深入理解时,可能依赖主流认知体系,导致推理中扭曲本土知识结构。然而,量化评估藏医文化偏见的工具仍严重缺失。为此,我们提出TreeProbe,首个围绕藏医本体树框架构建的文化偏见评测基准,包含4719个专家审核的条目,覆盖467种疾病和10个子任务。在代表性LLM上的实验表明,当前模型在藏医语境下表现受限,存在系统性的外部本体漂移。进一步分析显示,模型漂移方向取决于预训练数据构成及藏医与中医的表面相似性,可能趋向西医或中医推理。TreeProbe为开发语言包容、认知公平的医疗AI系统提供诊断工具。代码与数据可在匿名仓库https://anonymous.4open.science/r/TreeProbe/获取。

原文摘要 · Abstract (English)

Large language models are increasingly viewed as a potential means of mitigating global health inequities, yet their outputs often reflect dominant high-resource medical traditions and provide limited coverage of traditional medical knowledge systems. Tibetan medicine, one of the world's four major traditional medical systems, has an independent and highly structured theoretical framework. When models lack grounded understanding of Tibetan medicine, they may fall back on dominant epistemic systems and distort the native knowledge structure during reasoning. However, quantitative tools for evaluating cultural bias in Tibetan medicine remain largely absent. To address this gap, we introduce TreeProbe, the first cultural-bias benchmark organized around the native Tree of Medicine framework in Tibetan medicine. It contains 4,719 expert-adjudicated items covering 467 diseases and 10 subtasks along the three roots. Experiments on representative LLMs show that current models remain limited in native Tibetan medical contexts and exhibit systematic external ontology drift. Further analysis reveals that models diverge in whether they drift toward biomedical or TCM reasoning, shaped by pretraining data composition and surface resemblance between TCM and Tibetan medicine. TreeProbe provides a diagnostic benchmark for developing medical AI systems that are both linguistically inclusive and epistemically fair. Code and data are available in an anonymous repository at https://anonymous.4open.science/r/TreeProbe/.

藏医文化偏见大模型评测医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。