让AI自主设计符合硬件限制的模型，突破算力瓶颈。
Agentic evolution of physically constrained foundation models
- 用进化知识图谱引导搜索，实现有目标的结构演化
- 压缩2350亿参数模型，内存降75%仅损0.64%精度
- 适合需在有限硬件上部署大模型的研究者
人工智能正推动自动化科学发现，但现有通用智能体缺乏物理约束，常生成不兼容硬件的设计。本文提出一个物理约束的多智能体发现引擎，可自主设计符合硬件要求的计算系统。基于演化知识图谱整合过往科学创新，提取‘算法思维链’，将盲目随机搜索转化为定向结构演化。应用于基础模型部署的极端场景，该引擎演化出两种硬件感知压缩方法：Q-Enhance缓解密集模型长上下文精度损失，MoE-Salient-AQ在亚3比特条件下比最先进人工稀疏专家模型高出3.7%。通过高效的敏感度分析，成功将2350亿参数模型部署于双A100受限服务器，内存降低75%，精度仅下降0.64%。该工作将无约束组合搜索转化为知识驱动的自主设计，为机器驱动发现建立可扩展的软硬件协同范式。
原文摘要 · Abstract (English)
Artificial intelligence increasingly drives automated scientific discovery, yet contemporary generalist agents lack physical grounding, frequently hallucinating hardware-incompatible designs. Here, we present a physically grounded, multi-agent discovery engine that autonomously architects hardware-compliant computing systems. Anchored by an Evolutionary Knowledge Graph structuring past scientific innovations, the framework extracts an "algorithmic Chain-of-Thought" to transform blind stochastic search into directed structural evolution. Applied to the extreme testbed of foundation model deployment, the engine evolved two hardware-aware compression methodologies surpassing human-engineered heuristics: Q-Enhance mitigates long-context accuracy loss in dense models, and MoE-Salient-AQ outperforms state-of-the-art manual sparse Mixture-of-Experts designs by 3.7% at sub-3-bit regimes. Utilizing a bandwidth-efficient Sensitivity Profile, we successfully deployed a massive 235-billion-parameter model onto a constrained dual-A100 server, reducing memory requirements by 75% with a marginal 0.64% accuracy degradation. By transforming unconstrained combinatorial search into knowledge-driven autonomy, this establishes a scalable hardware-software co-design paradigm for machine-driven discovery within strict physical boundaries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。