提出解耦集成的架构搜索框架,让单次评估即可预测集成性能,提速百倍。
AgenticRS-EnsNAS: Ensemble-Decoupled Self-Evolving Architecture Search
- 用集成理论建立单模型评估与整体性能的映射关系,避免重复训练全部模型
- 在50-200个模型的集成场景下,单候选评估成本从O(M)降至O(1)
- 适用于工业级部署的连续、离散架构搜索,支持自动优化和大模型驱动探索
神经架构搜索(NAS)在工业系统中面临验证瓶颈:验证一个候选架构pi需对由M个模型组成的集成进行完整评估,导致每候选需耗时O(M),成本过高。本工作提出解耦集成架构搜索(Ensemble-Decoupled NAS),利用集成理论仅通过单学习器评估即可预测系统级性能。我们建立解耦集成理论,在同质性假设下给出单调改进的充分条件:当rho(pi) < rho(pi_old) - (M / (M - 1)) * (Delta E(pi) / sigma^2(pi))时,新架构比旧架构更优,其中Delta E、rho、sigma^2可通过轻量双学习器训练估算。该方法将单候选搜索成本从O(M)降低至O(1),仅对最终验证通过的候选保留O(M)部署开销。框架统一处理三类策略:(1)闭式优化用于可解析的连续架构(如CTR预测中的特征袋装);(2)受限可微优化处理不可解析连续架构;(3)大模型驱动的迭代接受式搜索处理离散架构。揭示了基础多样性提升与准确率提升两种独立优化机制,提供可操作的工业级NAS设计原则。所有推导均严谨,详细证明见附录。全面实证将在期刊扩展版中呈现。
原文摘要 · Abstract (English)
Neural Architecture Search (NAS) deployment in industrial production systems faces a fundamental validation bottleneck: verifying a single candidate architecture pi requires evaluating the deployed ensemble of M models, incurring prohibitive O(M) computational cost per candidate. This cost barrier severely limits architecture iteration frequency in real-world applications where ensembles (M=50-200) are standard for robustness. This work introduces Ensemble-Decoupled Architecture Search, a framework that leverages ensemble theory to predict system-level performance from single-learner evaluation. We establish the Ensemble-Decoupled Theory with a sufficient condition for monotonic ensemble improvement under homogeneity assumptions: a candidate architecture pi yields lower ensemble error than the current baseline if rho(pi) < rho(pi_old) - (M / (M - 1)) * (Delta E(pi) / sigma^2(pi)), where Delta E, rho, and sigma^2 are estimable from lightweight dual-learner training. This decouples architecture search from full ensemble training, reducing per-candidate search cost from O(M) to O(1) while maintaining O(M) deployment cost only for validated winners. We unify solution strategies across pipeline continuity: (1) closed-form optimization for tractable continuous pi (exemplified by feature bagging in CTR prediction), (2) constrained differentiable optimization for intractable continuous pi, and (3) LLM-driven search with iterative monotonic acceptance for discrete pi. The framework reveals two orthogonal improvement mechanisms -- base diversity gain and accuracy gain -- providing actionable design principles for industrial-scale NAS. All theoretical derivations are rigorous with detailed proofs deferred to the appendix. Comprehensive empirical validation will be included in the journal extension of this work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。