The ventral visual stream is known to be organized hierarchically, where early visual areas processing simplistic features feed into higher visual areas processing more complex features. Hierarchical convolutional neural networks (CNNs) were largely inspired by this type of brain organization and have been successfully used to model neural responses in different areas of the visual system. In this work, we aim to understand how an instance of these models corresponds to temporal dynamics of human object processing. Using representational similarity analysis (RSA) and various similarity metrics, we compare the model representations with two electroencephalography (EEG) data sets containing responses to a shared set of 72 images. We find that there is a hierarchical relationship between the depth of a layer and the time at which peak correlation with the brain response occurs for certain similarity metrics in both data sets. However, when comparing across layers in the neural network, the correlation onset time did not appear in a strictly hierarchical fashion. We present two additional methods that improve upon the achieved correlations by optimally weighting features from the CNN and show that depending on the similarity metric, deeper layers of the CNN provide a better correspondence than shallow layers to later time points in the EEG responses. However, we do not find that shallow layers provide better correspondences than those of deeper layers to early time points, an observation that violates the hierarchy and is in agreement with the finding from the onset-time analysis. This work makes a first comparison of various response features-including multiple similarity metrics and data sets-with respect to a neural network.

中文翻译：

---

对象处理中深层神经网络层与EEG测量之间的时间分辨对应关系。

已知腹侧视觉流是分层组织的，早期的视觉区域处理简单的特征会馈入较高的视觉区域，处理较复杂的特征。分层卷积神经网络（CNN）在很大程度上受到这种类型的大脑组织的启发，并已成功地用于对视觉系统不同区域的神经反应进行建模。在这项工作中，我们旨在了解这些模型的实例如何与人类对象处理的时间动态相对应。使用代表性相似性分析（RSA）和各种相似性度量，我们将模型表示与两个脑电图（EEG）数据集进行比较，其中两个数据集包含对共享的72张图像的响应。我们发现，对于两个数据集中的某些相似性指标，在层的深度与发生与大脑反应的峰值相关性的时间之间存在层次关系。但是，当在神经网络中跨层比较时，相关的开始时间并未以严格的分层方式出现。我们提出了两种其他方法，它们通过对CNN的特征进行最佳加权来改善已实现的相关性，并显示根据相似性度量，CNN的较深层比较浅的层对EEG响应中的较晚时间点的对应性更好。但是，我们发现，在较早的时间点上，浅层没有比深层提供更好的对应关系，违反层次结构且与开始时间分析的发现一致的观察结果。这项工作对各种响应特征（包括多个相似性度量和数据集）相对于神经网络进行了首次比较。