通过解耦优化与一致性约束，提升跨模态光流估计精度
Rethinking Unsupervised Cross-modal Flow Estimation: Learning from Decoupled Optimization and Consistency Constraint
- 分离处理模态差异与几何错位，分任务训练转换与光流网络
- 在无真值条件下实现高精度光流预测，优于现有无监督方法
- 适合做跨模态视觉任务的工程师和研究者参考
本文提出DCFlow，一种新型无监督跨模态光流估计框架，融合解耦优化策略与跨模态一致性约束。不同于以往仅依赖外观相似性隐式学习的方法，我们引入针对不同任务的监督机制，分别解决模态差异与几何错位问题。通过协同训练模态转换网络与光流估计网络实现该目标。为在无真实光流情况下获得可靠运动监督，设计了基于几何感知的数据合成流程及抗异常值损失函数。进一步引入跨模态一致性约束，联合优化两个网络，显著提升光流预测准确率。为评估，我们复用公开数据集构建了一个全面的跨模态光流基准。实验表明，DCFlow可适配多种光流网络，在无监督方法中达到领先性能。
原文摘要 · Abstract (English)
This work presents DCFlow, a novel unsupervised cross-modal flow estimation framework that integrates a decoupled optimization strategy and a cross-modal consistency constraint. Unlike previous approaches that implicitly learn flow estimation solely from appearance similarity, we introduce a decoupled optimization strategy with task-specific supervision to address modality discrepancy and geometric misalignment distinctly. This is achieved by collaboratively training a modality transfer network and a flow estimation network. To enable reliable motion supervision without ground-truth flow, we propose a geometry-aware data synthesis pipeline combined with an outlier-robust loss. Additionally, we introduce a cross-modal consistency constraint to jointly optimize both networks, significantly improving flow prediction accuracy. For evaluation, we construct a comprehensive cross-modal flow benchmark by repurposing public datasets. Experimental results demonstrate that DCFlow can be integrated with various flow estimation networks and achieves state-of-the-art performance among unsupervised approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。