arXiv:2512.08968cs.LG2025-12被引 1

用生物能量原理优化Transformer专家路由,大幅降低能耗。

StructuredDNA: A Bio-Physical Framework for Energy-Aware Transformer Routing

  • 基于语义能量最小化构建动态专家路由机制。
  • 在BioASQ上能耗密度降低97.7%,语义稳定性达0.998。
  • 适用于多领域、可扩展的节能架构,适合大规模模型研究者。

大型计算模型的快速扩展导致能源与算力成本急剧上升。受生物系统中结构与功能由低能耗构型自然形成启发,我们提出StructuredDNA——一种模块化、节能的稀疏架构框架。该框架以生物物理能量引导的路由层替代传统的密集型专家混合(Mixture-of-Experts)路由,通过语义能量最小化动态将输入分组为语义密码子,并选择单个专家以最小化包含凝聚力、不确定性与计算成本的全局能量函数。我们在专用(BioASQ)和开放域基准(WikiText-103)上验证了该方法。在BioASQ(K=50)上,能效利用密度(EUD)降低97.7%,语义稳定性指数(SSI)达0.998。在WikiText-103上,我们发现语义缩放定律:当专家粒度扩展至K=2048时,仍保持超过99%的能效。StructuredDNA建立了一种鲁棒、跨领域的稀疏计算范式,首次明确连接生物物理原理与Transformer专家路由,为未来节能、模块化、可扩展的计算系统指明方向。本文讨论该概念验证研究的局限性,并提出向更大模型、数据集与硬件平台扩展的路径。实现代码已开源:https://github.com/InnoDeep-repos/StructuredDNA。

原文摘要 · Abstract (English)

The rapid scaling of large computational models has led to a critical increase in energy and compute costs. Inspired by biological systems where structure and function emerge from low-energy configurations, we introduce StructuredDNA, a sparse architecture framework for modular, energy-aware Transformer routing. StructuredDNA replaces dense Mixture-of-Experts routing with a bio-physical, energy-guided routing layer based on semantic energy minimization. Inputs are dynamically grouped into semantic codons, and routing selects a single expert by minimizing a global energy functional that combines cohesion, uncertainty, and computational cost. We validate StructuredDNA on both specialized (BioASQ) and open-domain benchmarks (WikiText-103). On BioASQ (K = 50), we achieve a 97.7% reduction in Energy Utilization Density (EUD) and a Semantic Stability Index (SSI) of 0.998. We further demonstrate a Semantic Scaling Law on WikiText-103, showing that the architecture generalizes to open domains by scaling expert granularity (K = 2048) while maintaining more than 99% energy efficiency. StructuredDNA thus establishes a robust, domain-agnostic paradigm for future sparse computational frameworks. StructuredDNA provides an explicit link between bio-physical principles and sparse expert routing in Transformer architectures, and points toward future energy-aware, modular, and scalable computational systems. We discuss limitations of this proof-of-concept study and outline directions for scaling the approach to larger models, datasets, and hardware platforms. The StructuredDNA implementation is available at https://github.com/InnoDeep-repos/StructuredDNA .

Transformer节能计算稀疏路由生物启发

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。