无需文件头或完整文件，也能准确识别二进制片段类型。
MimeLens: Position-Agnostic Content-Type Detection for Binary Fragments
- 用随机偏移采样训练小型BERT模型，实现无位置依赖的文件类型识别。
- 在完整文件上比Magika高10.7个百分点，且可处理单个数据包或磁盘块。
- 适合恶意软件分析、取证等需碎片化处理的场景，但计算较慢。
文件类型分类在恶意软件筛选、取证提取、包检测和存储索引等任务中至关重要。现有方法如Google的Magika依赖完整文件和已知起始位置，无法处理实际任务中的片段输入，如单个数据包载荷、无头提取片段、随机磁盘块或分块上传。我们提出MimeLens，一种基于随机偏移采样训练的小型BERT类编码器，支持标准与短上下文两种变体。输入为任意位置的字节块，无需头信息或固定大小；输出为libmagic的125种MIME标签之一。在完整文件头部测试中，MimeLens相比Magika v1.1提升10.7个百分点（top-1）。它还能在单个流中UDP包及随机中间磁盘块上持续分类，性能是libmagic和Magika的两倍以上。代价是速度：在CPU上每样本慢一到两个数量级，但在消费级GPU或批量处理中可匹配。所有训练模型已发布于Hugging Face（mjbommar/mimelens-001-*）。
原文摘要 · Abstract (English)
File-type classification underlies many workflows like malware triage, forensic carving, packet inspection, and storage indexing. Learned systems such as Google's Magika assume whole-file access at a known offset, so they break on the inputs many of these tasks actually produce, like a single packet payload, a header-less carved fragment, a random disk block, or a chunked upload. We introduce MimeLens, a family of small BERT-style encoders pretrained on binary content from windows sampled at a uniformly random offset within each file, with no privileged head-of-file position, in standard- and short-context variants. A byte chunk goes in from anywhere in a file, no header needed and no fixed size; out comes one of libmagic's 125 MIME labels. On the clean head of complete files, MimeLens beats Magika v1.1 by +10.7 pp top-1 on libmagic-labeled data, and it keeps classifying where Magika cannot: from a single mid-stream UDP packet, and more than twice as accurately as libmagic and Magika on random mid-file disk blocks. The cost is latency: MimeLens runs roughly one to two orders of magnitude slower per sample on CPU than Magika, though it matches on consumer GPUs or in batch. All trained checkpoints are released on Hugging Face (mjbommar/mimelens-001-*).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。