arXiv:2608.21712cs.AIcs.MA2026-08

ATHENA用知识迁移加速医疗数据建模的模型搜索,省时又高效。

ATHENA: Knowledge-guided agentic neural architecture search for AutoFormer-based electronic health record modeling

论文配图:ATHENA: Knowledge-guided agentic neural architecture search for AutoFormer-based electronic health record modeling
图 1 · 摘自论文原文
  • 基于医院间共享的预训练超网,用继承子网络替代重复训练。
  • 在12次评估中9次超越基线,在30次搜索预算下表现最优。
  • 适合需要快速适配新医院数据的临床预测系统开发者。

基于Transformer的模型广泛用于电子健康记录(EHR)的临床预测,但其架构需人工调优,且最优配置随任务和医院而异。神经架构搜索(NAS)可自动化设计,但传统方法对Transformer-based EHR模型计算成本过高。近期基于大语言模型(LLM)的NAS虽减少人工设计,但各医院独立搜索,未复用架构知识。本文提出ATHENA(Agentic Transfer across Hospitals for EHR Neural Architecture Search),一种面向Transformer-based EHR建模的知识引导代理式NAS框架。ATHENA使用按医院预训练一次的权重共享超网,使候选架构以继承子网形式实例化并通过微调评估,而非独立预训练。引入两层跨医院架构先验:第一层根据任务描述符从源医院检索高性能架构;第二层利用基于SHAP的元回归估计组件影响。这些先验指导多代理LLM搜索,并结合目标医院验证反馈。在一处保留的OneFlorida+站点和一处外部MIMIC-IV站点上,针对六项临床预测任务的12次评估中,ATHENA在严格等计算条件下9次优于四个基线。采用通用预训练AutoFormer超网进行候选评估时,30次搜索预算下9次排名第一,且重复搜索结果更一致。ATHENA为减少Transformer-based EHR建模中的手动架构调优提供了实用方案。

原文摘要 · Abstract (English)

Transformer-based models are widely used for clinical prediction from electronic health records (EHRs), yet their architectures require manual tuning, and the optimal configuration may vary across tasks and hospitals. Neural architecture search (NAS) automates architecture design, but conventional methods are computationally costly for Transformer-based EHR models. Recent large language model (LLM)-guided NAS methods reduce manual search design but conduct each search independently, without reusing architecture knowledge across hospitals. In this study, we propose ATHENA (Agentic Transfer across Hospitals for EHR Neural Architecture Search), a knowledge-guided agentic NAS framework for Transformer-based EHR modeling. ATHENA uses a weight-sharing supernet that is pretrained once per hospital, allowing candidate architectures to be instantiated as inherited subnetworks and evaluated through fine-tuning rather than independent pretraining. It incorporates a two-layer cross-hospital architecture prior. The first layer retrieves high-performing architecture examples from source sites based on task descriptors, while the second estimates the effects of architectural components using SHapley Additive exPlanations (SHAP)-based meta-regression. These priors guide a multi-agent LLM search using validation feedback from the target hospital. Across six clinical prediction tasks evaluated at one held-out OneFlorida+ site and one external MIMIC-IV site, ATHENA significantly outperforms all four baselines in 9 of 12 site-task evaluations under a strict equal-compute comparison. Using a common pretrained AutoFormer supernet for candidate evaluation, ATHENA ranks first in 9 of 12 evaluations at a search budget of 30. It also shows more consistent architecture selection across repeated searches. ATHENA provides a practical approach for reducing manual architecture tuning in Transformer-based EHR modeling.

医疗AI神经架构搜索知识迁移Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。