通过分析嵌入维度波动，用非序列编码解码一致性检测异常
Forewarned is Forearmed: When Non-Sequential Embedding Turns Into an Anomaly Detector
- 利用编码解码一致性识别敏感嵌入维度
- 构建高精度异常检测器，准确率显著提升
- 适合关注多模态模型可靠性与鲁棒性的研究者
本文深入分析了非序列多模态句子级嵌入，重点研究SONAR模型。实验表明，某些嵌入维度对扰动敏感，可作为解码异常的指示信号。通过利用连续编码与解码之间的一致性，成功构建了一个高精度异常检测器。此外，我们还尝试对特定感兴趣维度进行修改以实现修正。本工作强调了理解与分析嵌入本身对于提升多模态表示可靠性的关键作用。
原文摘要 · Abstract (English)
This paper offers an in-depth analysis of non-sequential multimodal sentence-level embeddings, with a particular focus on the SONAR model. We demonstrate that certain embedding dimensions are sensitive to perturbations and can serve as indicators of decoding anomalies. By leveraging the consistency between successive encoding and decoding, we successfully build an accurate detector. Additionally, we explore modifying specific dimensions of interest to attempt to correct them. This work underscores the importance of understanding and analyzing the embeddings themselves to enhance the reliability of multimodal representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。