Many cognitive, sensory and motor processes have correlates in oscillatory neural source activity, which is embedded as a subspace in the recorded brain signals. Decoding such processes from noisy magnetoencephalogram/electroencephalogram (M/EEG) signals usually requires data-driven analysis methods. The objective evaluation of such decoding algorithms on experimental raw signals, however, is a challenge: the amount of available M/EEG data typically is limited, labels can be unreliable, and raw signals often are contaminated with artifacts. To overcome some of these problems, simulation frameworks have been introduced which support the development of data-driven decoding algorithms and their benchmarking. For generating artificial brain signals, however, most of the existing frameworks make strong and partially unrealistic assumptions about brain activity. This limits the generalization of results observed in the simulation to real-world scenarios. In the present contribution, we show how to overcome several shortcomings of existing simulation frameworks. We propose a versatile alternative, which allows for an objective evaluation and benchmarking of novel decoding algorithms using real neural signals. It allows to generate comparatively large datasets with labels being deterministically recoverable from the arbitrary M/EEG recordings. A novel idea to generate these labels is central to this framework: we determine a subspace of the true M/EEG recordings and utilize it to derive novel labels. These labels contain realistic information about the oscillatory activity of some underlying neural sources. For two categories of subspace-defining methods, we showcase how such labels can be obtained—either by an exclusively data-driven approach (independent component analysis—ICA), or by a method exploiting additional anatomical constraints (minimum norm estimates—MNE). We term our framework post-hoc labeling of M/EEG recordings. To support the adoption of the framework by practitioners, we have exemplified its use by benchmarking three standard decoding methods—i.e., common spatial patterns (CSP), source power-comodulation (SPoC), and convolutional neural networks (ConvNets)—wrt. Varied dataset sizes, label noise, and label variability. Source code and data are made available to the reader for facilitating the application of our post-hoc labeling framework.

中文翻译：

---

任意 M/EEG 记录的事后标记，用于神经解码方法的数据高效评估

许多认知、感觉和运动过程与振荡神经源活动相关，这些活动作为子空间嵌入到记录的大脑信号中。从嘈杂的脑磁图/脑电图 (M/EEG) 信号中解码此类过程通常需要数据驱动的分析方法。然而，对实验原始信号进行此类解码算法的客观评估是一个挑战：可用的 M/EEG 数据量通常是有限的，标签可能不可靠，并且原始信号经常受到伪影的污染。为了克服其中一些问题，引入了模拟框架，支持数据驱动解码算法及其基准测试的开发。然而，为了生成人工大脑信号，大多数现有框架对大脑活动做出了强有力且部分不切实际的假设。这限制了模拟中观察到的结果推广到现实世界场景的能力。在当前的贡献中，我们展示了如何克服现有模拟框架的几个缺点。我们提出了一种通用的替代方案，它允许使用真实的神经信号对新颖的解码算法进行客观的评估和基准测试。它允许生成相对较大的数据集，并且可以从任意 M/EEG 记录中确定性地恢复标签。生成这些标签的新颖想法是该框架的核心：我们确定真实 M/EEG 记录的子空间并利用它来派生新颖的标签。这些标签包含有关某些潜在神经源振荡活动的真实信息。对于两类子空间定义方法，我们展示了如何获得此类标签——要么通过专门的数据驱动方法（独立成分分析 - ICA），要么通过利用额外的解剖约束的方法（最小范数估计 - MNE）。我们将我们的框架称为 M/EEG 记录的事后标记。为了支持从业者采用该框架，我们通过对三种标准解码方法进行基准测试来举例说明其使用，即通用空间模式（CSP）、源功率共调制（SPoC）和卷积神经网络（ConvNets）。不同的数据集大小、标签噪声和标签变异性。源代码和数据可供读者使用，以促进我们的事后标签框架的应用。