让AI理解带优先级的多规则指令，提升复杂任务执行能力
PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis
- 构建四阶段数据生成框架，合成带优先级的多规则指令数据
- 仅用1万条数据，模型严格准确率从9.5%提升至30.1%
- 适用于需要精确执行多规则任务的AI系统，如智能评审与决策
现实中的多模态指令常包含重要性不一的多重要求，但现有训练数据仍将其简化为单一问答任务。本文通过规则理解研究这一差距，将模型视为规则执行者而非生成者：给定图像和带优先级的规则集，模型需逐条验证规则后输出综合判断。为此提出PRISM框架，包含人物-任务对生成、前缀引导规则集构建、质量过滤规则集及结构化验证轨迹生成四个阶段。进一步提出PRISM-Eval评估基准，其松散与严格指标基于固定标签的确定性匹配，无需推理时裁判模型。仅使用1万条合成样本，Qwen3-VL-4B在PRISM-Eval上的严格准确率从9.5%提升至30.1%，同时保持通用基准性能稳定，且效果可迁移至四种开源多模态大模型，涵盖密集与MoE架构，表明结构化规则监督是实现多规则、优先级感知多模态指令遵循的可扩展路径。
原文摘要 · Abstract (English)
Real-world multimodal instructions often bundle multiple requirements with unequal importance, yet most multimodal training data still reduce instruction following to answering one self-contained question. We study this gap through rubric comprehension, which casts the model not as a generator measured against rubrics but as an executor that follows them: given an image and a typed, prioritized rubric, the model must verify each rule before producing an overall judgment. To support this setting, we propose PRISM, a four-stage data synthesis framework that produces persona--task pairs, prefix-guided rule sets, quality-filtered rubrics, and structured verification traces. We further introduce PRISM-Eval, whose Loose and Strict metrics use deterministic matching against fixed labels and therefore require no inference-time judge model. With only 10K synthesized samples, PRISM lifts Qwen3-VL-4B from 9.5% to 30.1% Strict accuracy on PRISM-Eval while preserving average performance on general benchmarks, and the gains transfer to four additional open-source MLLMs across dense and MoE architectures, suggesting that structured rubric supervision is a scalable path toward multi-rule, priority-aware multimodal instruction following.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。