用CLIP和知识库实现语义图像传输，重在语义一致而非像素还原。
Knowledge-Base based Semantic Image Transmission Using CLIP
- 用CLIP提取图像语义特征，压缩后传输
- 接收端通过向量库匹配找回最相似图像
- 以类别一致为成功标准，适合语义通信研究
本文提出一种基于知识库（KB）的语义图像传输框架。接收端利用对比语言-图像预训练（CLIP）模型提取图像语义嵌入，构建基于Facebook AI相似性搜索（FAISS）的向量数据库。发送端先使用CLIP提取512维语义特征，再通过轻量神经网络压缩传输。接收端将信号恢复为512维特征，并在知识库中进行相似性匹配，检索语义最相近的图像。系统以传输图像与检索图像类别一致性作为语义传输成功的判定标准，而非传统峰值信噪比（PSNR）。该框架强调语义准确性，为语义感知通信系统提供新评估范式。在CIFAR100数据集上的实验验证了其有效性。
原文摘要 · Abstract (English)
This paper proposes a novel knowledge-Base (KB) assisted semantic communication framework for image transmission. At the receiver, a Facebook AI Similarity Search (FAISS) based vector database is constructed by extracting semantic embeddings from images using the Contrastive Language-Image Pre-Training (CLIP) model. During transmission, the transmitter first extracts a 512-dimensional semantic feature using the CLIP model, then compresses it with a lightweight neural network for transmission. After receiving the signal, the receiver reconstructs the feature back to 512 dimensions and performs similarity matching from the KB to retrieve the most semantically similar image. Semantic transmission success is determined by category consistency between the transmitted and retrieved images, rather than traditional metrics like Peak Signal-to-Noise Ratio (PSNR). The proposed system prioritizes semantic accuracy, offering a new evaluation paradigm for semantic-aware communication systems. Experimental validation on CIFAR100 demonstrates the effectiveness of the framework in achieving semantic image transmission.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。