This paper concentrates on the approximation power of deep feed-forward neural networks in terms of width and depth. It is proved by construction that ReLU networks with width $\mathcal{O}(\max \{d\lfloor N^{1/d}\rfloor ,N+2\})$ and depth $\mathcal{O}(L)$ can approximate a Hölder continuous function on ${[0,1]}^d$ with an approximation rate $\mathcal{O}\left(\lambda \sqrt{d}{(N^2{L}^2\ln N)}^{-\alpha /d}\right)$, where $\alpha \in (0,1]$ and $\lambda >0$ are Hölder order and constant, respectively. Such a rate is optimal up to a constant in terms of width and depth separately, while existing results are only nearly optimal without the logarithmic factor in the approximation rate. More generally, for an arbitrary continuous function f on ${[0,1]}^d$, the approximation rate becomes $\mathcal{O}\left(\sqrt{d}{\omega }_f\right({(N^2{L}^2\ln N)}^{-1/d}\left)\right)$, where ${\omega }_f(\cdot )$ is the modulus of continuity. We also extend our analysis to any continuous function f on a bounded set. Particularly, if ReLU networks with depth 31 and width $\mathcal{O}(N)$ are used to approximate one-dimensional Lipschitz continuous functions on $[0,1]$ with a Lipschitz constant $\lambda >0$, the approximation rate in terms of the total number of parameters, $W=\mathcal{O}(N^2)$, becomes $\mathcal{O}(\frac{\lambda }{W\ln W})$, which has not been discovered in the literature for fixed-depth ReLU networks.

本文重点研究深度前馈神经网络在宽度和深度方面的逼近能力。构造证明，具有宽度的 ReLU 网络$\mathcal{哦}(\mathrm{最大限度}\{d\lfloor N^{1/d}\rfloor ,N+2\})$ 和深度 $\mathcal{哦}(升)$ 可以近似一个 Hölder 连续函数 ${[0,1]}^d$ 以近似率 $\mathcal{哦}\left(\lambda \sqrt{d}{(N^2{升}^2\mathrm{输入}N)}^{-\alpha /d}\right)$， 在哪里 $\alpha \in (0,1]$ 和 $\lambda >0$分别是 Hölder 阶和常数。这样的速率在宽度和深度方面分别达到常数时是最佳的，而现有的结果只是在近似速率中没有对数因子的情况下接近最佳。更一般地，对于任意连续函数f on${[0,1]}^d$，近似率变为 $\mathcal{哦}\left(\sqrt{d}{\omega }_F\right({(N^2{升}^2\mathrm{输入}N)}^{-1/d}\left)\right)$， 在哪里 ${\omega }_F(\cdot )$是连续性的模数。我们还将我们的分析扩展到有界集合上的任何连续函数f。特别是，如果深度为 31 且宽度为 31 的 ReLU 网络$\mathcal{哦}(N)$ 用于逼近一维 Lipschitz 连续函数 $[0,1]$ 与 Lipschitz 常数 $\lambda >0$，就参数总数而言的近似率， $宽=\mathcal{哦}(N^2)$，变成 $\mathcal{哦}(\frac{\lambda }{宽\mathrm{输入}宽})$，在固定深度 ReLU 网络的文献中尚未发现。