EulerLoRA让低秩微调能估计不确定性，用少量参数生成多样预测。
EulerLoRA: Rank-Driven Jump Dynamics for Calibrated Parameter-Efficient Fine-Tuning
- 通过采样低秩适配器的结构化变化，实现随机性扩展。
- 仅用300万参数就达到与1000万参数集成模型相当的性能。
- 适合需要快速部署且关注预测可信度的场景。
低秩适配（LoRA）实现了参数高效的微调，但标准LoRA仅生成单一确定性模型，无法直接支持预测不确定性估计。我们提出EulerLoRA，一种LoRA的随机扩展，通过在共享低秩适配器的秩一组件上采样结构化扰动，生成多条预测轨迹，同时保持期望下的确定性变换。我们在CIFAR-10、CIFAR-100和HAM10000上的视觉变压器模型上评估EulerLoRA，还测试了在SVHN上的分布外检测。在这些基准上，EulerLoRA表现可媲美或优于强基线的LoRA-Ensemble。使用两个秩为20的适配器，EulerLoRA仅需约300万可训练参数，而一个秩8、16个适配器的LoRA-Ensemble约需1000万参数，相当于减少约69%的可训练参数。结果表明，少量共享适配器即可获得有效的预测多样性。
原文摘要 · Abstract (English)
Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning, but standard LoRA produces a single deterministic model and does not directly support predictive uncertainty estimation. We introduce EulerLoRA, a stochastic extension of LoRA that generates multiple predictive trajectories by sampling structured variations along the rank-one components of shared low-rank adapters, while preserving the deterministic LoRA transformation in expectation. We evaluate EulerLoRA with vision transformers on CIFAR-10, CIFAR-100, and HAM10000, together with out-of-distribution detection on SVHN. Across these benchmarks, EulerLoRA achieves comparable or improved performance relative to strong LoRA-Ensemble baselines. Using two rank-20 adapters, EulerLoRA requires approximately 3 million trainable adapter parameters, compared with about 10 million for a rank-8, 16-adapter LoRA-Ensemble, corresponding to roughly 69% fewer trainable parameters. These results show that useful predictive diversity can be obtained from a small number of shared adapters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。