用大模型直接生成约束优化的最优解集合,速度快效果好。
Large Language Models as Amortized Pareto-Front Generators for Constrained Bi-Objective Convex Optimization

- 用微调大模型直接输出连续决策向量的帕累托前沿
- 70亿参数模型在5类问题上达到95%以上最优解覆盖率
- 推理快至0.16秒/例,适合实时多目标决策场景
生成约束双目标连续优化的可行帕累托前沿是多准则决策的核心。现有方法通常依赖迭代标量化、进化搜索或特定问题求解器,需对每例重复优化。我们提出DIPS框架,将大语言模型微调为约束双目标凸优化的摊销帕累托前沿生成器。给定文本问题描述,DIPS直接输出有序的可行连续决策向量集,逼近帕累托前沿。为适配自回归语言建模,DIPS结合紧凑离散化方案、数值锚定的词元初始化及三阶段课程优化,逐步对齐结构有效性、可行性与帕累托质量。在五类约束双目标凸问题上,微调后的70亿参数模型相对于参考前沿实现95.29%至98.18%的归一化超体积比。借助vLLM加速推理,单例求解时间低至0.16秒,优于通用和推理型大模型基线。结果表明,大模型可作为高效的连续帕累托前沿近似生成器。
原文摘要 · Abstract (English)
Generating feasible Pareto fronts for constrained bi-objective continuous optimization is central to multi-criteria decision-making. Existing methods usually rely on iterative scalarization, evolutionary search, or problem-specific solvers, requiring repeated optimization for each instance. We introduce DIPS, an end-to-end framework that fine-tunes large language models as amortized Pareto-front generators for constrained bi-objective convex optimization. Given a textual problem description, DIPS directly outputs an ordered set of feasible continuous decision vectors approximating the Pareto front. To make continuous optimization compatible with autoregressive language modeling, DIPS combines a compact discretization scheme, Numerically Grounded Token Initialization for new numerical tokens, and Three-Phase Curriculum Optimization, which progressively aligns structural validity, feasibility, and Pareto-front quality. Across five families of constrained bi-objective convex problems, a fine-tuned 7B-parameter model achieves normalized hypervolume ratios of 95.29% to 98.18% relative to reference fronts. With vLLM-accelerated inference, DIPS solves one instance in as little as 0.16 seconds and outperforms general-purpose and reasoning LLM baselines under the evaluated setting. These results suggest that LLMs can serve as effective amortized generators for continuous Pareto-front approximation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。