在隐私保护下实现专家跟踪的高效算法,适应不同攻击者类型。
Tracking the Best Expert Privately
- 设计三类对抗下的差分隐私算法,覆盖随机漂移、静态和自适应攻击者。
- 在分布最多漂移S次时,期望动态后悔上界为O(√(ST log(NT)) + S log(NT)/ε)。
- 揭示隐私保护下静态与自适应攻击者的根本差异,适用于高隐私场景研究者。
我们为动态后悔(即跟踪最优专家)问题设计了差分隐私算法,应对三种自然类型的对手:分布漂移的随机对手、静态对手和自适应对手,并在所有情况下实现了亚线性后悔。特别地,在分布可能漂移S次的随机漂移对手下,我们给出一个ε-差分隐私算法,其期望动态后悔上界为O(√(ST log(NT)) + S log(NT)/ε),其中T为时间步长,N为专家数量。对于静态对手,我们通过将动态后悔最小化归约为静态后悔最小化,得到期望动态后悔上界为O(√(ST log(NT)) + S T^{1/3} log(T/δ) log(NT)/ε^{2/3}),此时S表示最优专家允许切换次数。最后,类似静态后悔情形,我们在动态设置中建立了静态与自适应对手的根本分离:当ε ≤ √(S/T)时,任何(ε, δ)-差分隐私算法在自适应对手下必须承受线性动态后悔;而当ε ≫ √(S/T)时,我们给出了一个ε-差分隐私算法,可在自适应对手下实现亚线性动态后悔。
原文摘要 · Abstract (English)
We design differentially private algorithms for the problem of prediction with expert advice under dynamic regret, also known as tracking the best expert. Our work addresses three natural types of adversaries, stochastic with shifting distributions, oblivious, and adaptive, and designs algorithms with sub-linear regret for all three cases. In particular, under a shifting stochastic adversary where the distribution may shift $S$ times, we provide an $ε$-differentially private algorithm whose expected dynamic regret is at most $O\left( \sqrt{S T \log (NT)} + \frac{S \log (NT)}ε\right)$, where $T$ and $N$ are the epsilon horizon and number of experts, respectively. For oblivious adversaries, we give a reduction from dynamic regret minimization to static regret minimization, resulting in an upper bound of $O\left(\sqrt{S T \log(NT)} + \frac{S T^{1/3}\log(T/δ) \log(NT)}{ε^{2/3}}\right)$ on the expected dynamic regret, where $S$ now denotes the allowable number of switches of the best expert. Finally, similar to static regret, we establish a fundamental separation between oblivious and adaptive adversaries for the dynamic setting: while our algorithms show that sub-linear regret is achievable for oblivious adversaries in the high-privacy regime $ε\le \sqrt{S/T}$, we show that any $(ε, δ)$-differentially private algorithm must suffer linear dynamic regret under adaptive adversaries for $ε\le \sqrt{S/T}$. Finally, to complement this lower bound, we give an $ε$-differentially private algorithm that attains sub-linear dynamic regret under adaptive adversaries whenever $ε\gg \sqrt{S/T}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。