It is known that any target function is realized in a sufficiently small neighborhood of any randomly connected deep network, provided the width (the number of neurons in a layer) is sufficiently large. There are sophisticated analytical theories and discussions concerning this striking fact, but rigorous theories are very complicated. We give an elementary geometrical proof by using a simple model for the purpose of elucidating its structure. We show that high-dimensional geometry plays a magical role. When we project a high-dimensional sphere of radius 1 to a low-dimensional subspace, the uniform distribution over the sphere shrinks to a gaussian distribution with negligibly small variances and covariances.

中文翻译：

---

任何目标函数都存在于任何足够宽的随机网络的邻域中：几何视角

众所周知，只要宽度（一层中的神经元数量）足够大，任何目标函数都可以在任何随机连接的深度网络的足够小的邻域中实现。关于这个惊人的事实有复杂的分析理论和讨论，但严谨的理论非常复杂。为了阐明其结构，我们使用一个简单的模型给出了基本的几何证明。我们展示了高维几何起着神奇的作用。当我们将半径为 1 的高维球体投影到低维子空间时，球体上的均匀分布收缩为高斯分布，其方差和协方差可以忽略不计。