Electroencephalography (EEG) signals elicited by multimodal stimuli can drive brain-computer interfaces (BCIs), and research has demonstrated that visual and auditory stimuli can be employed simultaneously to improve BCI performance. However, no studies have investigated the effect of multimodal stimuli in rapid serial visual presentation (RSVP) BCIs. The present study proposed a rapid serial multimodal presentation (RSMP) BCI that incorporates artificial facial images and artificial voice stimuli. To clarify the effect of audiovisual stimuli on the RSMP BCI, scrambled images and masked sounds were applied instead of visual and auditory stimuli, respectively. The findings indicated that the audiovisual stimuli improved performance of the RSMP BCI, and that P300 at Pz contributed to classification accuracy. Online accuracy of the BCI reached 85.7 ± 11.5 %. Taken together, these findings may aid in the development of better gaze-independent BCI systems.

多模态刺激引起的脑电图 (EEG) 信号可以驱动脑机接口 (BCI)，研究表明，视觉和听觉刺激可以同时使用，以提高 BCI 性能。然而，没有研究调查多模态刺激在快速串行视觉呈现 (RSVP) BCI 中的影响。本研究提出了一种快速串行多模态呈现 (RSMP) BCI，它结合了人工面部图像和人工语音刺激。为了阐明视听刺激对 RSMP BCI 的影响，分别应用了加扰图像和掩蔽声音代替视觉和听觉刺激。结果表明，视听刺激提高了 RSMP BCI 的性能，Pz 处的 P300 有助于提高分类准确性。BCI在线准确率达到85。7 ± 11.5 %。综上所述，这些发现可能有助于开发更好的与凝视无关的 BCI 系统。