arXiv:2507.02883q-bio.BMcs.LG2025-07被引 1

针对无序蛋白区域预测短板,构建新评测基准揭示模型真实风险。

DISPROTBENCH: Uncovering the Functional Limits of Protein Structure Prediction Models in Intrinsically Disordered Regions

  • 以无序区为核心,整合多模态数据并引入不确定性评估
  • 发现蛋白质互作预测在无序区性能骤降,药物设计相对稳健
  • 适合关注模型可靠性、生物功能推断与药物研发的研究者

内在无序区域(IDRs)在细胞功能中起关键作用,但现有蛋白质结构预测评测主要聚焦于折叠良好的结构域,忽视了真实生物场景中的三大挑战:蛋白质结构复杂性、可靠真值数据稀缺性,以及预测不确定性可能引发高风险下游失败(如药物发现、蛋白-蛋白互作建模和功能注释)。我们提出DisProtBench,一个以IDR为中心的评测基准,将预测不确定性显式纳入蛋白质结构预测模型(PSPMs)的评估。为应对结构复杂性和真值稀缺问题,我们构建并统一了一个大规模、多模态数据集,涵盖疾病相关IDRs、GPCR-配体互作及多聚体蛋白复合物。为评估预测不确定性,我们引入功能性不确定性敏感度(FUS)——一种基于预测不确定性的分层指标,用于量化下游任务在不确定性下的表现。通过该基准,我们系统评估了前沿PSPMs,揭示出明显的任务依赖性失败模式:蛋白-蛋白互作预测在IDRs中显著下降,而基于结构的药物发现则相对稳健。这些现象在传统全局准确率指标下难以察觉,后者会高估不确定性下的功能可靠性。我们已将基准和代码开源至https://github.com/Susan571/DisProtBench。

原文摘要 · Abstract (English)

Intrinsically disordered regions (IDRs) play central roles in cellular function, yet remain poorly evaluated by existing protein structure prediction benchmarks. Current evaluations largely focus on well-folded domains, overlooking three fundamental challenges in realistic biological settings: the structural complexity of proteins, the resulting low availability of reliable ground truth, and prediction uncertainty that can propagate into high-risk downstream failures, such as in drug discovery, protein-protein interaction modeling, and functional annotation. We present DisProtBench, an IDR-centric benchmark that explicitly incorporates prediction uncertainty into the evaluation of protein structure prediction models (PSPMs). To address structural complexity and ground-truth scarcity, we curate and unify a large-scale, multi-modal dataset spanning disease-relevant IDRs, GPCR-ligand interactions, and multimeric protein complexes. To assess predictive uncertainty, we introduce Functional Uncertainty Sensitivity (FUS), a novel prediction uncertainty-stratified metric that quantifies downstream task performance under prediction uncertainty. Using this benchmark, we conduct a systematic evaluation of state-of-the-art PSPMs and reveal clear, task-dependent failure modes. Protein-protein interaction prediction degrades sharply in IDRs, while structure-based drug discovery remains comparatively robust. These effects are largely invisible to standard global accuracy metrics, which overestimate functional reliability under prediction uncertainty. We have open-sourced our benchmark and the codebase at https://github.com/Susan571/DisProtBench.

蛋白结构预测无序区域不确定性评估药物发现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。