通过保持目标顺序关系提升深度回归性能
Improving Deep Regression with Tightness
- 用最优传输正则化确保特征空间中目标相似性
- 实验显示新方法在三个真实任务上显著提升精度
- 适合关注回归模型泛化能力的研究者
在深度回归中，保持目标与特征表示之间的序数关系能提升多种任务的性能。然而，这种优势的理论解释仍不明确。本文发现，保持序数关系可降低特征表示Z对目标Y的条件熵H(Z|Y)，而传统回归损失对此几乎无效。为此，我们提出一种基于最优传输的正则化方法，以保留目标间的相似关系，从而减小H(Z|Y)。此外，还引入简单高效的重复回归目标策略，同样旨在降低该熵值。在三个真实世界回归任务上的实验验证了所提方法的有效性。
原文摘要 · Abstract (English)
For deep regression, preserving the ordinality of the targets with respect to the feature representation improves performance across various tasks. However, a theoretical explanation for the benefits of ordinality is still lacking. This work reveals that preserving ordinality reduces the conditional entropy $H(Z|Y)$ of representation $Z$ conditional on the target $Y$. However, our findings reveal that typical regression losses do little to reduce $H(Z|Y)$, even though it is vital for generalization performance. With this motivation, we introduce an optimal transport-based regularizer to preserve the similarity relationships of targets in the feature space to reduce $H(Z|Y)$. Additionally, we introduce a simple yet efficient strategy of duplicating the regressor targets, also with the aim of reducing $H(Z|Y)$. Experiments on three real-world regression tasks verify the effectiveness of our strategies to improve deep regression. Code: https://github.com/needylove/Regression_tightness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。