arXiv:2512.00283cs.LGcs.AI2025-12中稿 · ICML

用自动化方法为生物数据设计专用神经网络架构,提升模型性能。

BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models

  • 基于神经架构搜索,系统探索适合生物数据的网络结构。
  • 发现的新架构在多类生物数据上表现更优,超越传统通用模型。
  • 提炼出可复用的设计原则,适合生物领域模型开发者参考。

基础模型已推动自然语言处理与计算机视觉等领域的变革。然而,现有生物学领域的工作多直接套用通用机器学习模型架构,未考虑生物数据特有的理化性质与结构特征,导致性能欠佳,难以捕捉长程依赖、稀疏信息及复杂的潜在“语法规则”。为此,本文提出 BioArc 框架,突破依赖直觉的架构设计,实现针对生物基础模型的系统化、自动化架构发现。该框架利用神经架构搜索(NAS),在大规模设计空间中评估多种架构,跨多个生物模态分析架构、分词方式与训练策略的相互作用。通过此大规模分析,识别出新型高性能架构,并提炼出一组经验性设计原则,指导未来模型开发。此外,为高效利用这些发现的优质架构,本文还提出并比较了多种架构预测方法,可快速为新生物任务推荐最优结构。整体工作为下一代生物领域专用与基础模型的构建提供了基础资源与可遵循的方法论。

原文摘要 · Abstract (English)

Foundation models have revolutionized various fields such as natural language processing (NLP) and computer vision (CV). While efforts have been made to transfer the success of the foundation models in general AI domains to biology, existing works focus on directly adopting the existing foundation model architectures from general machine learning domains without a systematic design considering the unique physicochemical and structural properties of each biological data modality. This leads to suboptimal performance, as these repurposed architectures struggle to capture the long-range dependencies, sparse information, and complex underlying ``grammars'' inherent to biological data. To address this gap, we introduce BioArc, a novel framework designed to move beyond intuition-driven architecture design towards principled, automated architecture discovery for biological foundation models. Leveraging Neural Architecture Search (NAS), BioArc systematically explores a vast architecture design space, evaluating architectures across multiple biological modalities while rigorously analyzing the interplay between architecture, tokenization, and training strategies. This large-scale analysis identifies novel, high-performance architectures, allowing us to distill a set of empirical design principles to guide future model development. Furthermore, to make the best of this set of discovered principled architectures, we propose and compare several architecture prediction methods that effectively and efficiently predict optimal architectures for new biological tasks. Overall, our work provides a foundational resource and a principled methodology to guide the creation of the next generation of task-specific and foundation models for biology.

神经架构搜索生物模型基础模型自动化设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。