通过正交基优化与空域去噪，提升条件表征学习的准确性与泛化能力。
Refine and Purify: Orthogonal Basis Optimization with Null-Space Denoising for Conditional Representation Learning
- 采用奇异值分解构建正交语义基，结合曲率截断实现自适应优化。
- 通过投影到无关子空间的零空间，有效抑制非目标语义干扰。
- 在聚类、分类和检索任务中表现优于现有方法，适合定制化表征场景。
条件表征学习旨在为特定任务提取针对性特征。近期研究将通用特征投影到由大语言模型生成的文本基张成的条件特征子空间，以获得条件表示。然而，此类方法存在两大局限：对子空间基敏感，易受子空间间干扰影响。为此，本文提出OD-CRL框架，融合自适应正交基优化（AOBO）与零空间去噪投影（NSDP）。具体而言，AOBO通过带曲率截断的奇异值分解构建正交语义基；NSDP通过将嵌入投影至无关子空间的零空间，抑制非目标语义干扰。在定制化聚类、分类与检索任务上的大量实验表明，OD-CRL达到新最优性能，具备优异泛化能力。
原文摘要 · Abstract (English)
Conditional representation learning aims to extract criterion-specific features for customized tasks. Recent studies project universal features onto the conditional feature subspace spanned by an LLM-generated text basis to obtain conditional representations. However, such methods face two key limitations: sensitivity to subspace basis and vulnerability to inter-subspace interference. To address these challenges, we propose OD-CRL, a novel framework integrating Adaptive Orthogonal Basis Optimization (AOBO) and Null-Space Denoising Projection (NSDP). Specifically, AOBO constructs orthogonal semantic bases via singular value decomposition with a curvature-based truncation. NSDP suppresses non-target semantic interference by projecting embeddings onto the null space of irrelevant subspaces. Extensive experiments conducted across customized clustering, customized classification, and customized retrieval tasks demonstrate that OD-CRL achieves a new state-of-the-art performance with superior generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。