用光子芯片解决大模型长文本推理的内存瓶颈，实现每步查询只需固定开销。
PRISM: Breaking the O(n) Memory Wall in Long-Context LLM Inference via O(1) Photonic Block Selection
- 通过光子广播与微环权重实现快速块选择，避免逐个扫描键值缓存。
- 在64K上下文长度下准确率100%，数据传输量减少16倍，能效提升10000倍。
- 适合需要超长上下文推理的场景，如法律、医疗文档分析。
长上下文大模型推理的瓶颈并非算力，而是每解码步骤需扫描整个键值缓存带来的O(n)内存带宽开销——这是任何算力扩展都无法突破的壁垒。尽管近期光子加速器在密集注意力计算中表现出色，但应用于长上下文时仍继承电子计算的O(n)内存扩展性问题。我们首次发现，真正关键的优化点在于粗粒度块选择步骤：一个内存受限的相似性搜索，用于决定加载哪些键值块。该任务结构上恰好匹配光子广播与加权范式——查询通过无源分束向所有候选扩散，签名近似静态（匹配电光微环调制），且仅需排名顺序（精度可放宽至4-6位）。关键在于，光子优势随上下文长度增长而增强：当上下文长度N增加时，电子扫描成本线性上升，而光子评估成本保持恒定。我们基于此提出PRISM（Photonic Ranking via Inner-product Similarity with Microring weights），采用薄膜铌酸锂（TFLN）构建相似性引擎。在Qwen2.5-7B上进行硬件受限的“针堆大海”评估，证明从4K到64K token均达到100%准确率（k=32），64K上下文下数据流量减少16倍。在实际上下文长度（n ≥ 4K）下，能量效率相比GPU基线提升四个数量级。
原文摘要 · Abstract (English)
Long-context LLM inference is bottlenecked not by compute but by the O(n) memory bandwidth cost of scanning the KV cache at every decode step -- a wall that no amount of arithmetic scaling can break. Recent photonic accelerators have demonstrated impressive throughput for dense attention computation; however, these approaches inherit the same O(n) memory scaling as electronic attention when applied to long contexts. We observe that the real leverage point is the coarse block-selection step: a memory-bound similarity search that determines which KV blocks to fetch. We identify, for the first time, that this task is structurally matched to the photonic broadcast-and-weight paradigm -- the query fans out to all candidates via passive splitting, signatures are quasi-static (matching electro-optic MRR programming), and only rank order matters (relaxing precision to 4-6 bits). Crucially, the photonic advantage grows with context length: as N increases, the electronic scan cost rises linearly while the photonic evaluation remains O(1). We instantiate this insight in PRISM (Photonic Ranking via Inner-product Similarity with Microring weights), a thin-film lithium niobate (TFLN) similarity engine. Hardware-impaired needle-in-a-haystack evaluation on Qwen2.5-7B confirms 100% accuracy from 4K through 64K tokens at k=32, with 16x traffic reduction at 64K context. PRISM achieves a four-order-of-magnitude energy advantage over GPU baselines at practical context lengths (n >= 4K).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。