NOVA让推荐系统架构演进更智能，自动避免无效修改并加速上线。
NOVA: A Verification-Aware Agent Harness for Architecture Evolution in Industrial Recommender Systems
- 基于多轮搜索生成候选架构，结合生产约束与语义验证。
- 在固定预算下有效通过率超50%，生产测试提升GMV达2.02%。
- 适合工业级推荐系统研发团队，尤其关注可复用诊断经验。
工业广告推荐系统通过架构调整持续优化，但生产迭代依赖专家，需协调模型结构、特征配置和交互模块，满足接口、资源与服务约束。AutoML受限于预定义搜索空间，通用编码代理仅验证可运行性，而非推荐系统特异性语义正确性。生成的可执行方案可能违反架构契约，且缺乏对语义诊断与评估结果的结构化复用，导致重复无效修改。本文提出NOVA——一种验证感知的代理框架，将生产架构演进建模为固定评估预算下的多轮搜索。每轮生成多个符合生产约束的候选方案，剔除语义违规项，并对有效方案进行本地测试与离线评估排序。跨轮次中，轨迹记忆整合语义诊断、本地测试结果与离线指标变化，形成演化方向与禁止模式以指导后续搜索。在相同最大离线评估预算下，NOVA在ScaleUp任务上达到53.3%的有效通过率，在Literature-to-Production任务上达51.7%。在覆盖超十亿用户广告系统5%流量的生产A/B测试中，选定的Literature-to-Production候选方案在三个主要pCVR目标上分别带来+1.25%、+1.70%、+2.02%的GMV增长，对应绝对pCVR偏差降低58.8%、66.7%、37.3%。
原文摘要 · Abstract (English)
Industrial advertising recommender systems are continually improved through architecture modifications, yet production iteration remains expert-intensive because coordinated changes to model topology, feature configuration, and interaction modules must satisfy strict interface, resource, and serving constraints. AutoML is limited to predefined search spaces, while generic coding agents verify runnability rather than recommender-specific semantic validity. Executable candidates may therefore violate architectural contracts, while the lack of structured reuse of semantic diagnostics and evaluation outcomes can lead to repeated invalid or ineffective modifications. We present NOVA, a verification-aware agent harness that organizes production architecture modification as multi-round search over concrete implementations within a fixed evaluation budget. At each round, NOVA generates multiple candidates under production constraints, rejects semantic violations, and ranks the valid survivors for local testing and offline evaluation. Across rounds, trajectory memory synthesizes semantic diagnostics, local-test outcomes, and offline metric changes into modification directions and forbidden patterns that guide subsequent search. Under the same maximum offline-evaluation budget for automated methods, NOVA achieves the highest effective pass rate, reaching 53.3% on ScaleUp and 51.7% on Literature-to-Production tasks. In a production A/B test covering 5% of traffic in an advertising system serving over one billion users, the selected Literature-to-Production candidate yields GMV gains of +1.25%, +1.70%, and +2.02% across three major pCVR objectives, with corresponding relative reductions in absolute pCVR bias of 58.8%, 66.7%, and 37.3%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。