The ability to learn new tasks and generalize to others is a remarkable characteristic of both human brains and recent artificial intelligence systems. The ability to perform multiple tasks simultaneously is also a key characteristic of parallel architectures, as is evident in the human brain and exploited in traditional parallel architectures. Here we show that these two characteristics reflect a fundamental tradeoff between interactive parallelism, which supports learning and generalization, and independent parallelism, which supports processing efficiency through concurrent multitasking. Although the maximum number of possible parallel tasks grows linearly with network size, under realistic scenarios their expected number grows sublinearly. Hence, even modest reliance on shared representations, which support learning and generalization, constrains the number of parallel tasks. This has profound consequences for understanding the human brain’s mix of sequential and parallel capabilities, as well as for the development of artificial intelligence systems that can optimally manage the tradeoff between learning and processing efficiency.

学习新任务和推广到他人的能力是人类大脑和最近人工智能系统的显着特征。同时执行多个任务的能力也是并行架构的一个关键特征，这在人脑中很明显，并在传统的并行架构中得到了利用。在这里，我们展示了这两个特征反映了支持学习和泛化的交互式并行性与通过并发多任务处理支持处理效率的独立并行性之间的基本权衡。尽管可能的并行任务的最大数量随着网络大小线性增长，但在现实情况下，它们的预期数量呈亚线性增长。因此，即使是对支持学习和泛化的共享表示的适度依赖，限制并行任务的数量。这对于理解人脑的顺序和并行能力的组合，以及开发可以优化管理学习和处理效率之间的权衡的人工智能系统具有深远的影响。