We study the power of deep neural networks (DNNs) with sigmoid activation function. Recently, it was shown that DNNs approximate any $d$-dimensional, smooth function on a compact set with a rate of order $W^{-p/d}$, where $W$ is the number of nonzero weights in the network and $p$ is the smoothness of the function. Unfortunately, these rates only hold for a special class of sparsely connected DNNs. We ask ourselves if we can show the same approximation rate for a simpler and more general class, i.e., DNNs which are only defined by its width and depth. In this article we show that DNNs with fixed depth and a width of order $M^d$ achieve an approximation rate of $M^{-2p}$. As a conclusion we quantitatively characterize the approximation power of DNNs in terms of the overall weights $W_0$ in the network and show an approximation rate of $W_0^{-p/d}$. This more general result finally helps us to understand which network topology guarantees a special target accuracy.

中文翻译：

---

通过具有 sigmoid 激活函数的深度神经网络逼近平滑函数

我们研究了具有 sigmoid 激活函数的深度神经网络 (DNN) 的威力。最近，研究表明 DNN 在紧致集合上以 $W^{-p/d}$ 的阶率逼近任何 $d$ 维的平滑函数，其中 $W$ 是网络中非零权重的数量$p$ 是函数的平滑度。不幸的是，这些比率仅适用于一类特殊的稀疏连接的 DNN。我们问自己是否可以为更简单和更通用的类显示相同的近似率，即仅由其宽度和深度定义的 DNN。在本文中，我们展示了具有固定深度和 $M^d$ 阶宽度的 DNN 实现了 $M^{-2p}$ 的近似率。作为结论，我们根据网络中的整体权重 $W_0$ 定量表征 DNN 的逼近能力，并显示 $W_0^{-p/d}$ 的逼近率。