SIAM Journal on Optimization, Volume 31, Issue 2, Page 1131-1157, January 2021.
We consider multilevel composite optimization problems where each mapping in the composition is the expectation over a family of randomly chosen smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient method where the approximate gradients are obtained via nested stochastic variance reduction. In order to find an approximate stationary point where the expected norm of its gradient mapping is less than $\epsilon$, the total sample complexity of our method is $O(\epsilon^{-3})$ in the expectation case and $O(N+\sqrt{N}\epsilon^{-2})$ in the finite-sum case where $N$ is the total number of functions across all composition levels. In addition, the dependence of our total sample complexity on the number of composition levels is polynomial, rather than exponential as in previous work.

中文翻译：

---

嵌套方差减少的多层复合随机优化

SIAM优化杂志，第31卷，第2期，第1131-1157页，2021年1月。
我们考虑了多层组合优化问题，其中组合中的每个映射都是对一系列随机选择的平滑映射的期望或某个有限数量的平滑映射的总和。我们提出了一种归一化的近端近似梯度方法，其中近似梯度是通过嵌套的随机方差减少获得的。为了找到其梯度映射的期望范数小于$ \ epsilon $的近似固定点，在期望的情况下，我们的方法的总样本复杂度为$ O（\ epsilon ^ {-3}）$在有限和情况下，O（N + \ sqrt {N} \ epsilon ^ {-2}）$其中$ N $是所有组合级别上函数的总数。另外，我们总样本复杂度对合成水平数量的依赖性是多项式的，而不是像以前的工作那样是指数级的。