arXiv:2603.03493q-bio.MNcs.LG2026-03

发现基因调控网络推断方法排名受评估方式影响,稳定性堪忧。

Quantifying Ranking Instability Across Evaluation Protocol Axes in Gene Regulatory Network Benchmarking

  • 设计诊断框架,分解评估协议变动对排名的影响
  • 不同评估条件导致方法排名互换率最高达32.1%
  • 揭示排名不稳源于方法区分能力变化,非数据基数问题

基准排名常被用于论证基因调控网络(GRN)推断方法的质量,但其在合理评估协议变动下的稳定性很少被检验。本文提出系统性诊断框架,量化协议迁移下的排名不稳定性,并提供分解工具,分离基线率效应与区分能力效应。基于三个成人组织的单细胞GRN基准数据及六种推断方法,我们测得四种协议轴上的成对排名反转率:候选集限制(16.3%,95%置信区间11.0至23.4%)、组织背景(19.3%)、参考网络选择(32.1%)和符号映射策略(0.0%)。置换零模型显示,观察到的反转率远低于随机预期(0.163对比零模型均值0.500),表明排名结构部分稳定但非不变。分解分析表明,反转主要由方法相对区分能力变化驱动,而非基线率膨胀,挑战了GRN基准中常见的隐含假设。本文建议采用稳定性感知的报告实践,并提供诊断工具以识别易发生排名反转的方法对。

原文摘要 · Abstract (English)

Benchmark rankings are routinely used to justify scientific claims about method quality in gene regulatory network (GRN) inference, yet the stability of these rankings under plausible evaluation protocol choices is rarely examined. We present a systematic diagnostic framework for measuring ranking instability under protocol shift, including decomposition tools that separate base rate effects from discrimination effects. Using existing single cell GRN benchmark outputs across three human tissues and six inference methods, we quantify pairwise reversal rates across four protocol axes: candidate set restriction (16.3 percent, 95 percent CI 11.0 to 23.4 percent), tissue context (19.3 percent), reference network choice (32.1 percent), and symbol mapping policy (0.0 percent). A permutation null confirms that observed reversal rates are far below random order expectations (0.163 versus null mean 0.500), indicating partially stable but non invariant ranking structure. Our decomposition reveals that reversals are driven by changes in the relative discrimination ability of methods rather than by base rate inflation, a finding that challenges a common implicit assumption in GRN benchmarking. We propose concrete reporting practices for stability aware evaluation and provide a diagnostic toolkit for identifying method pairs at risk of reversal.

基因调控评估稳定性方法比较

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。