CAPT模型实现跨数据集跨物种钙成像信号的通用建模与迁移。
CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

- 通过连续补丁标记与自回归训练,直接建模连续钙信号。
- 在小鼠、斑马鱼幼虫和线虫数据上迁移性能超越主流基线。
- 适用于跨物种神经动态预测与行为解码,适合神经科学通用模型研究者。
大规模钙成像为构建神经群体动态的基础模型提供了可能,但核心问题仍未解决:一个在特定记录集上预训练的模型能否泛化到新数据集、实验范式甚至不同物种?现有方法多针对特定任务设计,且仅在单一数据集评估,难以判断其学习表征是否可复用于新钙信号数据。为此,我们提出CAPT(连续自回归群体变压器),一种直接通过连续补丁标记策略建模连续钙信号,并采用自回归方式训练的模型,支持端到端预训练与下游任务适配。我们首先在大规模小鼠钙成像数据集上预训练CAPT,评估其在独立的小鼠、斑马鱼幼虫和秀丽隐杆线虫数据集上的迁移能力,这些数据由不同实验室采集。在迁移设置中,冻结预训练主干,仅更新适配模块。在神经群体预测与行为解码任务中,CAPT持续优于专用及通用基线模型。结合线虫数据中的NeuroPAL注释进行多模态分析显示,CAPT嵌入在不同数据集中形成共享功能空间,并捕捉与细胞身份相关的解剖结构。结果表明,连续自回归建模为构建通用神经基础模型开辟了简单路径,可跨数据集、实验范式与物种实现泛化。代码已公开于https://github.com/TSuXinH/CAPT。
原文摘要 · Abstract (English)
Large-scale calcium imaging has created an opportunity to build foundation-style models for neural population dynamics, but a central question remains unresolved: \textbf{whether a model pretrained on one collection of recordings can generalize to new datasets, experimental paradigms, and even species.} Existing approaches are often designed for specific tasks and evaluated on a single dataset, making it unclear whether their learned representations are reusable for new calcium trace datasets. To tackle this gap, we present \textbf{CAPT}, a \textbf{C}ontinuous \textbf{A}utoregressive \textbf{P}opulation \textbf{T}ransformer for calcium population dynamics. CAPT models continuous calcium traces directly through a continuous patch tokenization strategy and is trained autoregressively, enabling end-to-end pretraining and adaptation to diverse downstream tasks. We first pretrain CAPT on a large-scale mouse calcium imaging dataset and evaluate its transferability across independent mouse, larval zebrafish, and \textit{C. elegans} datasets collected by different laboratories. In these transfer settings, the pretrained backbone is frozen and only adaptation modules are updated. Across neural population forecasting and behavior decoding tasks, CAPT consistently outperforms specialized and general-purpose baselines. Alongside predictive performance, multimodal analyses using NeuroPAL annotations in \textit{C. elegans} datasets show that CAPT embeddings form a shared functional space across datasets and capture anatomical cell-identity-related structure. These results suggest that the continuous autoregressive modeling opens up possibilities for a simple route towards general-purpose neural foundation models for calcium imaging, which can generalize across datasets, experimental paradigms, and species. Code is available at https://github.com/TSuXinH/CAPT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。