通过提前排除无效动作,让网络控制更高效可靠。
Learning Not to Optimize: Physics-Informed Action-Space Reshaping for Intent-Based Network Control
- 先用中间信号筛选出不该优化的动作,缩小搜索空间。
- 小规模实例减少75.9%候选动作,保留90.8%接近最优的覆盖。
- 适合追求高效率与合规性的网络策略部署场景。
现代网络策略控制将意图映射为一系列部署-控制决策。传统贝尔曼风格的策略优化主要关注选择哪个动作,而约束通常通过惩罚、障碍或拉格朗日机制处理。我们观察到,在价值函数能准确判定最佳部署前,中间信号已可识别出大量应被排除的候选动作。这启发了一个互补方向:学习‘不优化’。在价值函数足够准确前,中间信号可能表明候选动作在状态-意图重标记下等价(商化),导致未来状态普遍更差(支配),或违反可执行的网络规律(残差筛选)。 LNOQRD{} 利用这些计算或学习到的信号作为影子过程,重塑主策略优化的动作空间,从而压缩行动域。我们证明了在显式等变性和单调性条件下,商化和支配是无损的,界定了前沿大小与排序成本,并量化了近似证书和原始估计带来的损失。实验表明, LNOQRD{} 在小实例上减少了75.9%的候选动作,同时保持90.8%接近最优的覆盖;在大规模实例上,实现了最高效用与意图满足度,最低硬性规则违规和生成后延迟,平均比基于候选的基线减少73.0%的开销。
原文摘要 · Abstract (English)
Modern network policy control maps intent to sequential placement-control decisions. Bellman-style policy optimization primarily asks which action to optimize, while constraints are commonly handled through penalty, barrier, or Lagrangian mechanisms. We observe that before a value function can certify the best deployment, intermediate signals may already identify many candidates that should be excluded from further optimization. This motivates a complementary direction: \emph{Learning Not to Optimize}. Before a value function is accurate enough to select the best placement-control decision, intermediate signals may already show that candidates are equivalent under state--intent relabeling (quotienting), lead to a uniformly worse future state (dominance), or violate executable network laws (residual screening). \LNOQRD{} uses these computed or learned signals as a shadow process to reshape the domain on which primal policy optimization is performed, thereby reducing the action space. We prove lossless quotienting and dominance under explicit equivariance and monotonicity conditions, bound frontier size and ranking cost, and quantify losses from approximate certificates and primal estimates. Experiments show that \LNOQRD{} reduces small-instance candidates by $75.9\%$ while retaining $90.8\%$ near-oracle coverage and, on large instances, achieves the highest utility and intent satisfaction, the lowest hard-law violation and post-generation latency, and a $73.0\%$ average reduction among candidate-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。