用多目标优化方法，在有限预算下平衡安全、不误拒和指令遵循。
MOSAIC: Multi-Objective Slice-Aware Iterative Curation for Alignment
- 基于失败分析动态调整数据混合比例与权重，实现闭环数据优化。
- 在100万令牌预算下，安全性能提升至4.67，误拒率保持4.41。
- 适合需要精细控制对齐质量的模型训练团队使用。
我们研究如何在固定监督微调预算下，同时平衡多轮安全对齐、对良性边界查询的低误拒率以及可验证约束下的指令遵循能力。提出MOSAIC（多目标切片感知迭代净化框架），一种基于统一L1-L3评估接口的闭环数据混合搜索框架。MOSAIC将切片级失败特征转化为可执行的数据操作，包括数据集级混合比例、桶级权重和聚焦标准。在相同基础模型上进行五轮独立微调、固定100万令牌预算条件下，内部XGuard得分从2.76提升至4.67，同时保持OrBench为4.41，IFEval为3.65。最终帕累托解在独立攻击、误拒和能力测试中均优于随机静态LoRA基线，表明结构化失败诊断可作为预算受限数据构建的有效控制信号。代码已公开于https://github.com/douyipu/mosaic。
原文摘要 · Abstract (English)
We study how to allocate a fixed supervised fine-tuning budget when three objectives must be balanced at once: multi-turn safety alignment, low over-refusal on benign boundary queries, and instruction following under verifiable constraints. We propose MOSAIC (Multi-Objective Slice-Aware Iterative Curation for Alignment), a multi-objective framework for closed-loop data mixture search built on a unified L1-L3 evaluation interface. MOSAIC turns slice-level failure profiles into executable data actions, including dataset-level mixture ratios, bucket-level weights, and focus criteria. Under a fixed 1M-token budget and five rounds of independent fine-tuning from the same base model, MOSAIC improves internal XGuard from 2.76 to 4.67 while keeping OrBench at 4.41 and IFEval at 3.65. The final Pareto solution also generalizes better than a random static LoRA baseline on independent attack, over-refusal, and capability tests, suggesting that structured failure diagnosis can serve as a practical control signal for budgeted data construction. Code is available at https://github.com/douyipu/mosaic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。