提升元学习在长尾任务下的鲁棒性，理论与实践双突破。
Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning
- 将分布鲁棒策略转化为极大极小优化问题，构建斯塔克尔伯格均衡解
- 推导出在长尾风险下的泛化界，证明方法可有效降低极端任务风险
- 适用于多模态大模型，显著增强快速适应中的鲁棒性，适合系统级安全场景
元学习是大模型时代的重要范式，任务分布鲁棒性已成为真实场景中的关键考量。近期研究已验证长尾任务风险最小化对快速适应鲁棒性的提升效果。本文进一步开展理论分析与实践改进：将分布鲁棒策略简化为极大极小优化问题，以斯塔克尔伯格均衡作为解概念，并估计收敛速率；在存在长尾风险的条件下，推导泛化界，建立与分位数估计的联系，并实际优化该策略。大量实验验证了所提方法的有效性及其在多模态大模型上的可扩展性，显著提升了鲁棒性。
原文摘要 · Abstract (English)
Meta learning is a promising paradigm in the era of large models and task distributional robustness has become an indispensable consideration in real-world scenarios. Recent advances have examined the effectiveness of tail task risk minimization in fast adaptation robustness improvement \citep{wang2023simple}. This work contributes to more theoretical investigations and practical enhancements in the field. Specifically, we reduce the distributionally robust strategy to a max-min optimization problem, constitute the Stackelberg equilibrium as the solution concept, and estimate the convergence rate. In the presence of tail risk, we further derive the generalization bound, establish connections with estimated quantiles, and practically improve the studied strategy. Accordingly, extensive evaluations demonstrate the significance of our proposal and its scalability to multimodal large models in boosting robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。