提出URECA算法,解决语义代码搜索中分布偏移的适应难题
URECA: The Chain of Two Minimum Set Cover Problems exists behind Adaptation to Shifts in Semantic Code Search
- 基于最小集合覆盖重构最小熵问题,揭示表征解耦关系缺失缺陷
- 在CoSQA查询偏移场景下实现少样本自适应的SOTA性能
- 适合关注模型鲁棒性与分布外适应的研究者
适应旨在使模型学习训练分布之外的模式。通常,该适应被形式化为最小熵问题,但其存在固有缺陷——初始化偏移级联现象。通过勒贝格积分扩展最小熵问题与最小集合覆盖问题的关系,发现最小熵机制忽略了解耦表征间的关联,导致初始化偏移级联。基于此分析,提出一种新型聚类算法:基于并查集的递归聚类算法(URECA)。URECA利用解耦表征间关系实现高效聚类,其更新规则基于阈值可更新平稳假设(Thresholdly-Updatable Stationary Assumption),作为平稳假设的简化版本,可无误差传输解耦表征。同时采用模拟技巧提升聚类效率。广泛评估表明,URECA在多种类型偏移下的少样本适应中均取得一致性能提升,并在查询偏移场景的CoSQA任务中达到当前最优表现。
原文摘要 · Abstract (English)
Adaptation is to make model learn the patterns shifted from the training distribution. In general, this adaptation is formulated as the minimum entropy problem. However, the minimum entropy problem has inherent limitation -- shifted initialization cascade phenomenon. We extend the relationship between the minimum entropy problem and the minimum set cover problem via Lebesgue integral. This extension reveals that internal mechanism of the minimum entropy problem ignores the relationship between disentangled representations, which leads to shifted initialization cascade. From the analysis, we introduce a new clustering algorithm, Union-find based Recursive Clustering Algorithm~(URECA). URECA is an efficient clustering algorithm for the leverage of the relationships between disentangled representations. The update rule of URECA depends on Thresholdly-Updatable Stationary Assumption to dynamics as a released version of Stationary Assumption. This assumption helps URECA to transport disentangled representations with no errors based on the relationships between disentangled representations. URECA also utilize simulation trick to efficiently cluster disentangled representations. The wide range of evaluations show that URECA achieves consistent performance gains for the few-shot adaptation to diverse types of shifts along with advancement to State-of-The-Art performance in CoSQA in the scenario of query shift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。