首个病毒基因组基础模型评测基准,揭示模型泛化与安全风险
ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks

- 构建涵盖18种场景的病毒基因组评测体系,评估模型生物理解与潜在安全风险
- 发现模型在进化和时间迁移下性能下降,生成结果统计合理但功能无效
- 证明预训练数据多样性比参数量更重要,轻量模型跨数据提升67.5%
核酸序列是生物系统的基本遗传单元,病毒基因组分析对医学进步至关重要。尽管生物基础模型(特别是核酸基础模型,NFMs)取得进展,但该领域缺乏统一的病毒基因组评测标准,难以推动社区发展并保障生物安全。为此,我们提出ViroBench,首个专为病毒场景设计的综合性、大规模基准。ViroBench从生物学理解与潜在生物安全风险两个维度,覆盖4类任务中的18种多样化场景,对66个不同架构的NFMs进行评估。结果显示:第一,NFMs在系统发育与时间偏移下表现出性能退化,说明其外推能力弱;第二,生成任务中统计似然与生物功能有效性脱钩,存在潜在生物安全风险;第三,受控消融实验表明,预训练数据的分类多样性优于参数规模——一个轻量基线模型在多样数据上训练后,性能相较原模型提升67.5%。ViroBench提供可解释、诊断性的评估与可复现的测量框架,助力未来研究。数据集与代码已开源于https://github.com/QIANJINYDX/ViroBench。
原文摘要 · Abstract (English)
Nucleotide sequences constitute the fundamental genetic basis of biological systems, rendering viral genomic analysis critical for biomedical advancement. Despite progress in biological foundation models, specifically nucleotide foundation models (NFMs), the field lacks a unified standard for viral genomics to facilitate community development and enforce biosecurity constraints. To address this, we introduce ViroBench, the first comprehensive and large-scale benchmark specifically designed for NFMs in viral settings. ViroBench evaluates models across two critical dimensions: biological understanding and latent biosecurity risk, covering 18 diverse scenarios within 4 task types. Extensive evaluation of 66 NFMs across diverse architectures yields three critical conclusions. Firstly, NFMs exhibit a performance degradation in biological understanding under phylogenetic and temporal shifts, indicating weak extrapolation capabilities. Secondly, generation tasks reveal a decoupling between statistical likelihood and biological functional validity, posing latent biosecurity risks. Thirdly, controlled ablation studies reveal that taxonomic diversity in pretraining data outweighs parameter scale. Specifically, a lightweight baseline trained on diverse data achieves a 67.5% performance gain over its original model. Overall, ViroBench provides interpretable, diagnostic evaluations and a reproducible measurement framework for future research on viral nucleotide foundation models. The datasets and code are publicly available at https://github.com/QIANJINYDX/ViroBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。