提出可审计的记忆机制，实现选择性保留与验证删除。
Forgetful Attention: An Auditable Support-Vector Memory for Selective Retention and Verified Deletion
- 用支持向量机思想设计注意力门控，系数为零的键保留，正数的键激活。
- 1200次删除重拟合试验中1199次成功，删除后读出结果不变，误差极小。
- 适合需要数据可验证删除的场景，如医疗记录、隐私保护系统。
可审计记忆需明确约定：哪些输出被保留，相对于哪个参考解，以及在何种更新下。本文提出支持向量注意力（SV-Attention），一种基于单类支持向量数据描述（SVDD）的门控机制，其系数参与读出计算。零系数键为保留状态，正系数键为激活状态。移除保留键而不重新求解，可保持当前读出结果不变；维持删除则更新现有求解器状态，目标是在相同系数上限C下对新保留键进行拟合。在高斯分布、冗余数据、MIMIC-IV及学习得到的键上共执行1200次浮点64位删除/重拟合试验，1199次成功完成。声明探针的最大门控得分偏差中位数介于4.5e-13至5.7e-7之间；在参考CPU上，重拟合速度比维持删除慢24至223倍。确定性示例显示，当前保留状态不能保证未来新增后的等价性。在匹配标记数量下，稀有组召回率分别为0.861（本文方法）和0.319（基准注意力质量代理，即H2O风格）。在独立通道的重症监护室控制实验中，血氧饱和度低于90%定义事件，但所有选择器均不包含血氧数据；事件小时保留率分别为0.464（本文方法）与0.225（RBF密度基线）。另有一批近似方案支持端到端训练；在322万参数规模下，七组种子平均最优验证比特/字符为2.178，显著低于2.383（p=0.001）。合同为瞬时且固定系数上限，未来安全流式处理与通用性能仍待探索，更大步长运行表明优化速度更慢。
原文摘要 · Abstract (English)
Auditable memory requires a precise contract: which output is preserved, relative to which reference solve, and across which updates. We introduce Support Vector Attention (SV-Attention), a one-class support vector data description (SVDD) gate whose coefficients enter the readout. Zero-coefficient keys are reserve; positive-coefficient keys are active. Removing a reserve token without re-solving preserves the current readout. Maintained deletion, which updates the existing solver state, targets a fresh retained-key fit under the same coefficient cap C. Across 1,200 fp64 deletion/refit trials on Gaussian, redundant, MIMIC-IV, and learned keys, 1,199 complete. Median maximum gate-score discrepancy over declared probes ranges from 4.5e-13 to 5.7e-7, and fresh refitting is 24-223 times slower than maintained deletion on the reference CPU. A deterministic example shows that current reserve status does not guarantee equivalence after future admissions. At matched token counts, rare-group recall is 0.861 versus 0.319 for an oracle attention-mass proxy (H2O-style). In a held-out-channel ICU control, SpO2 below 90 percent defines events but SpO2 is excluded from every selector; event-hour retention is 0.464 versus 0.225 for an RBF-density baseline. A separate batched approximation supports end-to-end training; at 3.22M parameters, seven paired seeds yield mean best-validation bits per character of 2.178 versus 2.383 (p=0.001). The contracts are point-in-time and fixed-C; future-safe streaming and general-purpose performance remain open, and larger fixed-step runs suggest slower optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。