让不懂代码的专家也能快速搭建自定义Transformer模型。
AttentionSmithy: A Modular Framework for Rapid Transformer Development and Customization
- 将Transformer拆成可复用模块，支持快速组合与实验
- 在基因分类任务中实现超95%准确率，验证实际效果
- 适合无编程基础的研究者加速跨领域模型开发
Transformer架构虽已重塑人工智能应用，但对缺乏底层实现经验的领域专家而言仍难定制。本文提出AttentionSmithy，一个模块化软件包，将注意力机制、前馈网络、归一化层和位置编码等核心组件解耦为可复用构建块，使用户无需大量编码即可快速原型设计与评估Transformer变体。框架支持四种位置编码策略，并可集成神经架构搜索实现自动化设计。我们在资源受限条件下复现了原始Transformer，通过融合位置编码优化了翻译性能。此外，在基因特异性建模中，该框架实现细胞类型分类准确率超过95%。多个案例表明，AttentionSmithy能有效降低框架开发门槛，推动跨领域研究加速。
原文摘要 · Abstract (English)
Transformer architectures have transformed AI applications but remain complex to customize for domain experts lacking low-level implementation expertise. We introduce AttentionSmithy, a modular software package that simplifies transformer innovation by breaking down key components into reusable building blocks: attention modules, feed-forward networks, normalization layers, and positional encodings. Users can rapidly prototype and evaluate transformer variants without extensive coding. Our framework supports four positional encoding strategies and integrates with neural architecture search for automated design. We validate AttentionSmithy by replicating the original transformer under resource constraints and optimizing translation performance by combining positional encodings. Additionally, we demonstrate its adaptability in gene-specific modeling, achieving over 95% accuracy in cell type classification. These case studies highlight AttentionSmithy's potential to accelerate research across diverse fields by removing framework implementation barriers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。