The link between mind, brain, and behavior has mystified philosophers and scientists for millennia. Recent progress has been made by forming statistical associations between manifest variables of the brain (e.g., electroencephalogram [EEG], functional MRI [fMRI]) and manifest variables of behavior (e.g., response times, accuracy) through hierarchical latent variable models. Within this framework, one can make inferences about the mind in a statistically principled way, such that complex patterns of brain–behavior associations drive the inference procedure. However, previous approaches were limited in the flexibility of the linking function, which has proved prohibitive for understanding the complex dynamics exhibited by the brain. In this article, we propose a data-driven, nonparametric approach that allows complex linking functions to emerge from fitting a hierarchical latent representation of the mind to multivariate, multimodal data. Furthermore, to enforce biological plausibility, we impose both spatial and temporal structure so that the types of realizable system dynamics are constrained. To illustrate the benefits of our approach, we investigate the model’s performance in a simulation study and apply it to experimental data. In the simulation study, we verify that the model can be accurately fitted to simulated data, and latent dynamics can be well recovered. In an experimental application, we simultaneously fit the model to fMRI and behavioral data from a continuous motion tracking task. We show that the model accurately recovers both neural and behavioral data and reveals interesting latent cognitive dynamics, the topology of which can be contrasted with several aspects of the experiment.

思维，大脑和行为之间的联系使哲学家和科学家们迷惑了数千年。通过分层的潜在变量模型在大脑的显性变量（例如，脑电图[EEG]，功能性MRI [fMRI]）和行为的显性变量（例如，响应时间，准确性）之间形成统计关联，已经取得了最新进展。在此框架内，人们可以以统计学上有原则的方式对思维进行推理，从而使大脑与行为关联的复杂模式驱动推理过程。但是，以前的方法在链接功能的灵活性方面受到限制，事实证明，这种方法无法理解大脑表现出的复杂动态。在本文中，我们提出了一种数据驱动的，一种非参数方法，它允许将思维的层次化潜在表示适合多变量，多模态数据，从而产生复杂的链接功能。此外，为了增强生物学上的合理性，我们强加了时空结构，以便限制可实现的系统动力学的类型。为了说明我们方法的好处，我们在模拟研究中研究了模型的性能，并将其应用于实验数据。在仿真研究中，我们验证了该模型可以准确地拟合到仿真数据，并且可以很好地恢复潜在的动力学。在实验应用中，我们同时将模型拟合到fMRI和来自连续运动跟踪任务的行为数据。