通过治理增强的混合微调，让多语言小样本模型更准更稳。
Governance-Aware Hybrid Fine-Tuning for Multilingual Large Language Models
- 混合使用低秩更新与正交变换，分层融合并加单位约束保稳定。
- 在XNLI和FLORES上优于主流方法，校准性更好且跨语言平衡。
- 轻量数据治理可叠加增益，适合算力有限的多语言应用。
我们提出一种面向多语言、低资源场景的治理感知混合微调框架。核心算法结合梯度对齐的低秩更新与分层混合的结构化正交变换，并在部分子层引入单位约束以稳定深层优化。同时配合轻量级无标签数据治理步骤（包括语言识别、近似重复项去除、质量过滤），在严格计算预算下兼顾准确性、概率校准与跨语言一致性。在XNLI和FLORES数据集上，该方法持续优于强基线PEFT模型，保持方向平衡并提升概率校准性能（见表II、III）。对轻量拼写变体更具鲁棒性（见表IV），且简单治理步骤可叠加增益（见表V）。训练开销测量显示小幅增加，成本-质量权衡良好（见表VI及图2）。结果表明，结合实用数据治理，混合与单位约束的PEFT为资源受限下的多语言适配提供了稳定且可行的路径。
原文摘要 · Abstract (English)
We present a governance-aware hybrid fine-tuning framework for multilingual, low-resource adaptation of large language models. The core algorithm combines gradient-aligned low-rank updates with structured orthogonal transformations through layer-wise mixing and introduces unitary constraints in selected sub-layers to stabilize deep optimization. In tandem with lightweight, label-free data governance steps, including language identification, near-duplicate removal, and quality filtering, the framework targets accuracy, calibration, and cross-language parity under tight compute budgets. Across XNLI and FLORES, the hybrid approach delivers consistent gains over strong PEFT baselines while maintaining directional balance and improving probability calibration, as shown in Tables II and III. It is more resilient to lightweight orthographic variants, as shown in Table IV, and benefits additively from simple governance steps, as shown in Table V. Training footprint measurements indicate modest overhead and a favorable cost-quality frontier, as shown in Table VI and Figure 2. Together, these results show that hybrid and unitary PEFT provide a stable and accessible path to resource-efficient multilingual adaptation when paired with practical data governance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。