Complexity is an indisputable, well-known, and broadly accepted feature of the brain. Despite the apparently obvious and widely-spread consensus on the brain complexity, sprouts of the single neuron revolution emerged in neuroscience in the 1970s. They brought many unexpected discoveries, including grandmother or concept cells and sparse coding of information in the brain. In machine learning for a long time, the famous curse of dimensionality seemed to be an unsolvable problem. Nevertheless, the idea of the blessing of dimensionality becomes gradually more and more popular. Ensembles of non-interacting or weakly interacting simple units prove to be an effective tool for solving essentially multidimensional and apparently incomprehensible problems. This approach is especially useful for one-shot (non-iterative) correction of errors in large legacy artificial intelligence systems and when the complete re-training is impossible or too expensive. These simplicity revolutions in the era of complexity have deep fundamental reasons grounded in geometry of multidimensional data spaces. To explore and understand these reasons we revisit the background ideas of statistical physics. In the course of the 20th century they were developed into the concentration of measure theory. The Gibbs equivalence of ensembles with further generalizations shows that the data in high-dimensional spaces are concentrated near shells of smaller dimension. New stochastic separation theorems reveal the fine structure of the data clouds. We review and analyse biological, physical, and mathematical problems at the core of the fundamental question: how can high-dimensional brain organise reliable and fast learning in high-dimensional world of data by simple tools? To meet this challenge, we outline and setup a framework based on statistical physics of data. Two critical applications are reviewed to exemplify the approach: one-shot correction of errors in intellectual systems and emergence of static and associative memories in ensembles of single neurons. Error correctors should be simple; not damage the existing skills of the system; allow fast non-iterative learning and correction of new mistakes without destroying the previous fixes. All these demands can be satisfied by new tools based on the concentration of measure phenomena and stochastic separation theory. We show how a simple enough functional neuronal model is capable of explaining: i) the extreme selectivity of single neurons to the information content of high-dimensional data, ii) simultaneous separation of several uncorrelated informational items from a large set of stimuli, and iii) dynamic learning of new items by associating them with already "known" ones. These results constitute a basis for organisation of complex memories in ensembles of single neurons.

中文翻译：

---

小神经集成体在高维大脑中的不合理有效性。

复杂性是大脑无可争议的，广为接受的特征。尽管在大脑复杂性方面有明显的共识和广泛的共识，但是在1970年代，神经科学领域出现了单一神经元革命的萌芽。他们带来了许多意想不到的发现，包括祖母或概念细胞以及大脑中信息的稀疏编码。在很长一段时间的机器学习中，著名的维数诅咒似乎是一个无法解决的问题。然而，维数祝福的想法逐渐变得越来越流行。非相互作用或弱相互作用的简单单元的集合被证明是解决本质上是多维且显然难以理解的问题的有效工具。这种方法对于大型传统人工智能系统中的错误的单次（非迭代）校正特别有用，并且在无法进行完全重新训练或成本太高的情况下，这种方法特别有用。复杂性时代的这些简单性革命具有深层的根本原因，其基础是多维数据空间的几何形状。为了探索和理解这些原因，我们重新审视了统计物理学的背景思想。在20世纪的过程中，它们发展成为测量理论的集中地。进一步综合的合奏的吉布斯等值表明，高维空间中的数据集中在较小维的壳附近。新的随机分离定理揭示了数据云的精细结构。我们审查并分析生物，物理，基本问题的核心是数学和数学问题：高维大脑如何通过简单工具在高维数据世界中组织可靠且快速的学习？为了应对这一挑战，我们概述并建立了一个基于数据统计物理学的框架。审查了两个关键的应用以例证该方法：一次纠正智能系统中的错误以及在单个神经元集合中出现静态和关联记忆。纠错器应该很简单；不损害系统的现有技能；允许快速的非迭代学习和纠正新错误，而不会破坏以前的修复程序。通过基于测量现象集中和随机分离理论的新工具可以满足所有这些需求。我们展示了一个足够简单的功能性神经元模型如何能够解释：i）单个神经元对高维数据信息内容的极端选择性； ii）同时从大量刺激中分离出几个不相关的信息项，以及iii ），通过将新商品与“已知”商品相关联来动态学习。这些结果构成了在单个神经元集合中组织复杂记忆的基础。