arXiv:2605.05054cs.CVcs.AI2026-05

提出解耦径向与角度动态的流匹配方法,提升视觉语言模型少样本适应性能。

Direct Product Flow Matching: Decoupling Radial and Angular Dynamics for Few-Shot Adaptation

论文配图:Direct Product Flow Matching: Decoupling Radial and Angular Dynamics for Few-Shot Adaptation
图 1 · 摘自论文原文
  • 将特征分解为径向和角度子流形,实现两者独立演化
  • 在11个基准上达到少样本适配新纪录
  • 适合需要快速适配新数据集的研究者

近期的流匹配(FM)方法通过建模跨模态对齐为连续多步流,提升了视觉语言模型的少样本适应能力。本文指出,现有方法受预训练跨模态特征中不兼容几何先验的限制,导致适应性能受限。从极坐标分解视角分析发现三个被忽视的缺陷:1)角度动力学失真:径向-角度耦合导致角度子流形速度不均,引发回归训练困难与额外截断误差;2)径向动力学忽略:特征归一化丢弃模态置信度,无法区分分布外与分布内数据,且丢失关键径向动态;3)上下文无关的无条件流:预训练跨模态特征提取中的数据集特异性信息损失无法恢复。为此,我们提出扭曲积流匹配(WP-FM),一个统一的黎曼框架,通过引入恒定扭曲度量,推导出直接积流匹配(DP-FM),构建解耦的柱状流形(即直接积流形)。DP-FM实现径向独立演化与恒速角度测地线传输,有效消除角度动力学失真并保持径向一致性。同时,通过条件化于预训练VLM的隐藏状态,引入分类器自由引导,注入缺失的数据集特异性信息。在11个基准上的大量实验表明,DP-FM在多步少样本适应任务上达到新最优性能。

原文摘要 · Abstract (English)

Recent flow matching (FM) methods improve the few-shot adaptation of vision-language models, by modeling cross-modal alignment as a continuous multi-step flow. In this paper, we argue that existing FM methods are inherently constrained by incompatible geometric priors on pre-trained cross-modal features, resulting in suboptimal adaptation performance. We first analyze these methods from a polar decomposition perspective (i.e., radial and angular sub-manifolds). Under this new geometric view, we identify three overlooked limitations in them: 1) Angular dynamics distortion: The radial-angular coupling induces non-uniform speed on the angular sub-manifold, leading to regression training difficulty and extra truncation errors. 2) Radial dynamics neglect: Feature normalization discards modality confidence, failing to distinguish out-of-distribution and in-distribution data, and abandoning crucial radial dynamics. 3) Context-agnostic unconditional flow: Dataset-specific information loss during pre-trained cross-modal feature extraction remains unrecovered. To resolve these issues, we propose warped product flow matching (WP-FM), a unified Riemannian framework that reformulates alignment on a warped product manifold. Within this framework, we derive direct product flow matching (DP-FM) by introducing a constant-warping metric, which yields a decoupled cylindrical manifold (i.e., direct product manifold). DP-FM enables independent radial evolution and constant-speed angular geodesic transport, effectively eliminating angular dynamics distortion while preserving radial consistency. Meanwhile, we incorporate classifier-free guidance by conditioning the flow on the pre-trained VLMs' hidden states to inject missing dataset-specific information. Extensive results across 11 benchmarks have demonstrated that DP-FM achieves a new state-of-the-art for multi-step few-shot adaptation.

流匹配少样本适应视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。