提出更优的差分隐私在线学习算法,突破时间依赖瓶颈。
Improved Regret in Stochastic Decision-Theoretic Online Learning under Differential Privacy
- 设计新算法实现无时间依赖的上界,仅对数级依赖动作数
- 在弱化设定下证明上下界匹配,达Θ(log K / ε)最优率
- 解决开放问题部分,适合关注隐私学习理论的研究者
Hu 和 Mehta(2024)提出了一个开放问题:在K个动作、T轮的随机决策论在线学习中,ε-差分隐私下的最优实例相关率是多少?此前已知的上界和下界分别为O(log K / Δ_min + log K log T / ε) 和 Ω(log K / Δ_min + log K / ε),其中Δ_min是最佳与次佳动作间的差距。本文部分解决该问题,提出两项新成果:首先,给出改进的上界O(log K / Δ_min + log²K / ε),该界不依赖T,且仅对数级依赖K;其次,引入较弱的确定性设定(接收损失向量为确定性),在此设定下,直接应用原设定的分析与算法仍导致额外对数因子。通过创新分析,证明了该设定下上下界均为Θ(log K / ε),达到一致。
原文摘要 · Abstract (English)
Hu and Mehta (2024) posed an open problem: what is the optimal instance-dependent rate for the stochastic decision-theoretic online learning (with $K$ actions and $T$ rounds) under $\varepsilon$-differential privacy? Before, the best known upper bound and lower bound are $O\left(\frac{\log K}{Δ_{\min}} + \frac{\log K\log T}{\varepsilon}\right)$ and $Ω\left(\frac{\log K}{Δ_{\min}} + \frac{\log K}{\varepsilon}\right)$ (where $Δ_{\min}$ is the gap between the optimal and the second actions). In this paper, we partially address this open problem by having two new results. First, we provide an improved upper bound for this problem $O\left(\frac{\log K}{Δ_{\min}} + \frac{\log^2K}{\varepsilon}\right)$, which is $T$-independent and only has a log dependency in $K$. Second, to further understand the gap, we introduce the \textit{deterministic setting}, a weaker setting of this open problem, where the received loss vector is deterministic. At this weaker setting, a direct application of the analysis and algorithms from the original setting still leads to an extra log factor. We conduct a novel analysis which proves upper and lower bounds that match at $Θ(\frac{\log K}{\varepsilon})$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。