用历史知识图谱动态优化神经网络设计，高效找到近优解。
Beyond Model Base Retrieval: Weaving Knowledge to Master Fine-grained Neural Network Design
- 构建编辑效应图谱，融合历史任务知识指导搜索
- 在33个任务中26次达最优性能，且搜索成本可控
- 适合需要快速迭代高精度模型的工程研发人员
为新任务设计高性能神经网络需权衡优化质量与搜索效率。现有方法要么计算开销大，要么依赖静态检查点导致性能不足。本文将细粒度结构修改带来的性能提升建模为编辑效应证据，并基于过往任务构建证据图谱。提出M-DESIGN框架，通过检索增强的模型精炼机制，动态编织历史证据以发现近优修改路径。其自适应检索机制可快速校准不同来源编辑效应证据的迁移性。为应对分布外变化，引入预测性任务规划器，从多跳证据中外推收益，降低对全量存储的依赖。基于包含67,760个图神经网络、覆盖22个数据集的模型知识库，实验表明M-DESIGN在严格预算下33个任务中有26次达到搜索空间最佳性能。
原文摘要 · Abstract (English)
Designing high-performance neural networks for new tasks requires balancing optimization quality with search efficiency. Current methods fail to achieve this balance: neural architectural search is computationally expensive, while model retrieval often yields suboptimal static checkpoints. To resolve this dilemma, we model the performance gains induced by fine-grained architectural modifications as edit-effect evidence and build evidence graphs from prior tasks. By constructing a retrieval-augmented model refinement framework, our proposed M-DESIGN dynamically weaves historical evidence to discover near-optimal modification paths. M-DESIGN features an adaptive retrieval mechanism that quickly calibrates the evolving transferability of edit-effect evidence from different sources. To handle out-of-distribution shifts, we introduce predictive task planners that extrapolate gains from multi-hop evidence, thereby reducing reliance on an exhaustive repository. Based on our model knowledge base of 67,760 graph neural networks across 22 datasets, extensive experiments demonstrate that M-DESIGN consistently outperforms baselines, achieving the search-space best performance in 26 out of 33 cases under a strict budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。