多目标提示优化中，文本梯度会因任务冲突而失效，影响模型判断力。
When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges
- 用文本梯度分解多目标反馈，但共享信息越多，任务专注度越低
- 联合优化多标准导致相关性下降0.085（0.305→0.220）
- 发现两类失效模式：优化时梯度稀释与推理时指令干扰
为特定问题定制LLM评判器通常需同时优化多个评估标准。现有文本梯度方法可自动处理单一评判标准，但生成的是自然语言批评而非数值向量，因此多任务学习中的冲突解决工具（如PCGrad、MGDA）无法适用。本文将TextGrad扩展至多目标场景，测试了四种文本梯度优化器的分解模式，通过改变损失、梯度和优化器LLM间共享的跨目标信息量。结果发现，当梯度LLM需联合提供多标准反馈时，其任务专注度从9.0降至3.7（满分10分），下降59%。此外，将单目标优化后的指令简单合并成单一提示，会导致斯皮尔曼等级相关系数从0.305降至0.220（-0.085）。研究揭示两类独立失效模式：优化阶段的梯度稀释与推理阶段的指令干扰，共同限制了基于文本反馈的多目标评判器优化设计空间。
原文摘要 · Abstract (English)
Customizing an LLM judge to a specific problem or domain often involves optimizing its prompt across multiple evaluation criteria simultaneously. Textual gradient methods automate this for a single judge criterion, however they produce natural-language critiques, not numerical vectors. Thus, the conflict-resolution toolkit of multi-task learning (PCGrad, MGDA) does not apply to this multi-objective textual gradient setting. We extend TextGrad to the multi-objective setting and test four decomposition modes of textual gradient optimizers by varying how much cross-objective information the loss, gradient and optimizer LLMs share. We find the gradient's task-focus drops by 59% (9.0 to 3.7 out of 10) when the gradient LLM must provide feedback on multiple criteria jointly. Separately, we observe that naively combining single-objective optimized instructions into a single prompt degrades Spearman rho from 0.305 to 0.220 (-0.085). These results identify two separable failure modes: optimization-time gradient dilution and inference-time instruction interference, which together constrain the design space for multi-objective judge optimization using textual feedback.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。