Introduction

Vestibular disorders affect between 0.7% and 15% of the general pediatric population (Gioacchini, Alicandri-Ciufelli, Kaleci, Magliulo, & Re, 2014). Among these, deaf and hard of hearing (D/HOH) children are more often affected, with around one-third of deaf children having associated vestibular loss (VL) (with the prevalence of this association varying depending on the type of vestibular testing) (Cushing, Papsin, Rutka, James, & Gordon, 2008; De Kegel, Maes, Baetens, Dhooge, & Van Waelvelde, 2012; Verbecque et al., 2017). Despite this well-known association, VL in children is likely underestimated (Rine, 2009), mainly because children compensate for the vestibular loss quicker than adults, and complain less due to insufficient language abilities and a lack of normative frame of reference in order to know that they have an impairment. This underestimation of VL in children has likely led to the lack of studies in the literature that have investigated the impact of VL on children's cognitive development. This is in contrast to some literature of VL in adults, which has reported subjective and objective cognitive difficulties (Edwards, 2007).

Since Beritoff (1965) first explored impaired navigation abilities in children with VL, only a few studies have investigated cognition in children with VL. These studies have tended to focus on delays in global motor development (De Kegel et al., 2012; Maes, De Kegel, Van Waelvelde, & Dhooge, 2014; Rine, 2009) and altered dynamic visual acuity (DVA) with possible less-efficient reading abilities (Braswell & Rine, 2006; Rine & Braswell, 2003). On the contrary, some studies on adults with VL have reported specific subjective and objective cognitive impairments in visuospatial working memory, mental rotation, selective attention and space orientation, and/or a decrease in quality of life (Agrawal, Ward, & Minor, 2013; Brandt et al., 2005; Candidi et al., 2013; Enloe & Shields, 1997; Lacroix et al., 2016; Péruch et al., 2011; Popp et al., 2017; Redfern, Talkowski, Jennings, & Furman, 2004; Schautzer, Hamilton, Kalla, Strupp, & Brandt, 2003). Complementary, experimental animal studies of VL and artificial vestibular stimulation (galvanic, caloric, or rotational) on healthy human participants have reported similar cognitive changes. For example, visuospatial working memory was altered in rodents with experimental VL (Baek, Zheng, Darlington, & Smith, 2010; Besnard et al., 2012; Russell, Horii, Smith, Darlington, & Bilkey, 2003). Also, artificial vestibular stimulation in healthy human control participants can modify spatial perception (Ferrè, Longo, Fiori, & Haggard, 2013), body schema (Lopez, Schreyer, Preuss, & Mast, 2012) or self-centered mental imagery (Deroualle, Borel, Devèze, & Lopez, 2015).

The reported studies on adults tended to use classical neuropsychological measures such as the Corsi-Block task (Popp et al., 2017) or line bisection task (Ferrè et al., 2013). Some research also used novel computerized techniques such as virtual mazes (Schautzer et al., 2003), virtual reality mental rotation task - trough third-person perspective task - (Deroualle et al., 2015) or computerized attention reaction time (Redfern et al., 2004). However, in the few studies that have been conducted on children with VL, computerized reaction time or automatic data recording have not been routinely used. Therefore, our aim was to develop a computerized test battery that could be used to evaluate visuospatial abilities in children.

Complementary to traditional paper-and-pencil neuropsychological tasks conventionally used in clinics, computerized measures allow the measurement of additional data such as reaction time, which tend to be less influenced by participant subjective bias and experimenter errors in manual data recording (Gur et al., 2001; Jagaroo, 2009). Computerized measures also have other advantages such as the maintenance of a standardized procedure for each testing session across patients (Claessen, van der Ham, & van Zandvoort, 2014; Jagaroo, 2009), the calculation of several scores during the same time frame and an improved experimenter-to-patient relationship due to automatic scoring that allows more time to be spent with the patient (Vaes et al., 2015). Although computerized neuropsychological testing is now more frequent and more developed, most current tests use a computer and screen display rather than interfaces using a tablet display. At the technological level, the growing development of innovative and interactive tablet displays allow a similar experience to that of paper-and-pencil measures. Recent developments of computerized tests for adults have been developed using tablet displays. These include the Visuospatial Neglect Test Battery for evaluating hemineglect in stroke patients (Vaes et al., 2015), the e-Corsi for measuring visuospatial working memory (Claessen et al., 2014), the Sleep-2-Peak Psychomotor Vigilance Test (PVT) (Brunet, Dagenais, Therrien, Gartenberg, & Forest, 2017), the Cognitive Assessment for diagnosis of Dementia (iPad version; CADi) (Onoda et al., 2013) and the National Center for Geriatrics and Gerontology Functional Assessment Tool (NCGG-FAT) (Makizako et al., 2012). Despite these developments, few tests using interactive tablet technology exist for cognitive evaluation in children. Some test batteries have used computer screen interfaces such as the Test of Everyday Attention for Children, Second Edition (TEA-Ch2; (Manly, Anderson, Crawford, George, & Robertson, 2016) or the Kinderen Test of Attentional Performance (KiTAP, Zimmermann, Gondan, & Fimm, 2005), but cognitive assessment of children using tablets just begins to emerge now. Some of the traditional assessments such as the Weschler Intelligence scales have recently been transferred to a tablet display using the “Q-interactive” system (Pearson, 2017). Also, new measures are currently being developed such as the assessment of cognitive and motor function allowing cross-cultural comparisons (Pitchford & Outhwaite, 2016) or the RED-App, an application that allows assessments in a classroom setting (Bignardi, Dalmaijer, & Astle, 2020; Dalmaijer et al., 2019)

The objective of our research here was to develop a new tablet-based computerized test battery to measure and better understand the cognition of children with VL. The Visuo-Spatial Abilities Diagnosis Test (VSAD) targets the four cognitive domains often associated to adult vestibular cognition: visuospatial working memory, mental rotation, selective attention, and space orientation. In this first stage, we aim to demonstrate concurrent validity of the VSAD relative to traditional paper-and-pencil neuropsychological tasks by comparing the results of the control participants group in both tasks. We then demonstrate the test–retest reliability of the new tasks using two testing sessions within a 1-month interval. Finally, we test the discriminant validity between children with VL and healthy averaged age-matched children.

Methods

Materials

Hardware and software

We used the Metrisquare DiagnosIS software (www.metrisquare.net) to develop and run the VSAD. Dual-screen technology allowed the use of an electronic pen display on a 13-inch Cintiq HD Pen tablet linked to a computer through a VGA connector and a USB port. This dual-screen presentation allowed the experimenter to observe the patient’s performance and to provide feedback (if necessary) during the testing. The software can run on any computer with a minimal configuration of a dual-core i5 or i7 processor and at least 4 GB RAM. The Metrisquare DiagnosIS software provides a user design interface where we implemented the stimuli specifically designed for our tasks using a Cartesian space (with ‘x’ referring to position along the long axis of the tablet, ‘y’ for the short axis, and ‘z’ for the pressure; see Vaes et al. (2015) for a more complete description of the system on a DTU-2231 Wacom tablet). Coded fields, additional plugins and scoring scripts allowed the registration of participants’ raw responses relative to the tablet calibration. After the testing, the software generates a comma-separated-value file or a clinical-type report with automatic calculated scores and screenshot of participants’ performances allowing data analysis through Excel or SPSS software. For the present study, and to facilitate experimenter control of stimuli presentation during the testing of younger children, the experimenter had to click on a “Next page” button for the initiation and continuation of subsequent stimuli in the task. This was used for all tasks except for the visuospatial working memory task and the mental rotation that required an automatic item presentation.

Computerized tasks

To measure visuospatial working memory, mental rotation, selective attention, and space orientation, we developed six different tasks (each explained in detail in the following paragraphs). Prior to development, we performed pilot testing on an independent sample of five children with the six tasks adapted to paper-and-pencil versions in order to ensure that the children understood the tasks and to verify suitable item complexity. The final computerized tasks were presented randomly to the children who participated in the present study. Before each recording, an example of each of the tasks was provided to ensure that the children correctly understood the tasks.

The visuospatial working memory task used a block-tapping task inspired by the Corsi Blocks task (Corsi, 1972) and by the computerized versions of the block-tapping tasks by Claessen et al. (2014). To create a child-friendly version of this task, we created a story where a lady pirate lost her parrot and the child had to find it. The parrot could appear in sequence in several green circles consecutively on the tablet screen and the child was asked to reproduce the same sequence of “parrot jumps”. The sequence length increased progressively; each sequence had two levels and the game automatically proceeded to the next level each time one of the two levels was succeeded. The game automatically stopped if the two sequences of a same level failed. The games had two versions, one forward and one backward, where the child had to reproduce the sequence in the same or reverse order. The software automatically registered the dependent variables of longest series (span) recalled and the averaged time by correct item in both the forward and backward directions. Figure 1 shows an example of the task.

Fig. 1
figure 1

Visuospatial working memory task. The child must reproduce the sequence of the parrot “jumps” in the same or contrary order

Full size image

For the mental rotation task, we created another child-friendly story where a knight could not recognize his shield, and the child had to help the knight to find it. Different shields designed with Adobe Illustrator software were implemented in Metrisquare DiagnoseIS software. In the task, an example of the knight’s shield appeared above different shields that were rotated, and the child had to recognize and point to the knight’s correct shield. The software automatically presented the next stimuli after each answer was registered. We manipulated task complexity by using the elements presented within the shield design (for example, in Fig. 2, the relative position of star and the diamond). For the easier items, the distractor items consisted of two elements, whereas for more difficult items, more elements were used. The number of shields also increase progressively during the task. The software automatically registered the dependent variables of total number of correct responses, and the averaged time by correct item.

Fig. 2
figure 2

Mental rotation task (performances of healthy control participant, 16 years old)

Full size image

To measure selective attention, we designed three tasks with different levels of complexity (Figs. 3, 4, and 5). The Space Rockets Cancellation Task displayed a target model of a space rocket at the top of the screen, and then an array of 104 space rockets (eight rows, with 13 items in each row), containing 16 randomly placed target rockets (four by quadrant). The rockets were all the same height, but had different characteristics (antennae, ladder, different numbers of windows and wings), all created with Adobe Photoshop. The child was required to point or cross the targets using the pen on the tablet display. A child-friendly story was used where the child had to help an astronaut find their space rocket (the target model). The target was always a space rocket with two windows, two wings, one ladder, and one antenna.

Fig. 3
figure 3

Space Rocket cancellation task (performances of one child with VL, 11 years old, profoundly deaf. The child omitted one target on the upper left side of space and also cancelled one distractor)

Full size image
Fig. 4
figure 4

House cancellation task (performances of one child with VL, 8 years old, with a severe deafness. The child did one omission on the bottom right of space)

Full size image
Fig. 5
figure 5

Cat cancellation task (performances of one child with VL, 8 years old, with a severe deafness. The child did one omission on the bottom right of space)

Full size image

The Houses Cancellation Task displayed 36 houses on the tablet screen (six rows and six columns), designed with Adobe Illustrator software. From the 36 houses, 24 were on fire. A child-friendly story was used to explain that the child was a fireman and he/she had to find and point to the safe houses (without flames, three by quadrant).

Finally, The Cats Cancellation Task was inspired from the Bells test (Gauthier, Dehaut, Joanette, & Yves Joanette, 1989). In this task, the child was asked to help a grandmother find her lost cat. Twenty-four cats, designed with Adobe Photoshop, were presented among 72 distractors (suns, trees and flowers; 98 shapes in total), presented randomly across the tablet screen (six by quadrant). The child was instructed to point to all the cats with the pen. For the three-cancellation tasks, the dependent variables automatically registered by the software were the total number omissions and errors and the time of task completion (from the beginning of the stimuli presentation to when the child indicated that they had found all the targets).

For the space orientation task, the child had to complete 12 different mazes (designed with Adobe Illustrator) by drawing a continuous line on the tablet from the entrance of the maze to the exit without touching the walls and/or turning back. Six pairs of mazes were provided, each with a similar level of complexity (same number of dead-ends, layout, and same length of the correct exit pathway), and presented either with simple or larger thickness lines (1 or 8 points). Before starting the task, a child-friendly story explained that the child had to help a racing car driver find the correct pathway in the maze. The story explained that the child had to drive as fast as possible, but also cautiously planning the pathway and not driving back and/or having an accident by touching the maze wall. This task was inspired by the classical paper-and-pencil Porteus Mazes (Porteus, 1950) and the LABY 5-12 (Marquet-Doléac, Soppelsa, & Albaret, 2010) tasks that measure space orientation and executive function (impulsivity, planning and delay aversion). For each succeed maze (when the exit was reached), the software automatically recorded the dependent variables of total length of the drawn path, total number of errors (cut lines/crashes), total planning time (time between the maze presentation on the screen and the time when the child started to draw), the total execution time (time between when the child started to draw and the child arrived at the maze exit), total time (planning plus execution time) and the total number of pen lifts. For the present study, in the test–retest analyses, we created a second version of the task where the mazes were rotated 180 degrees. As we found no statistically significant differences between these two versions, we used the average score of responses in the two versions when analyzing discriminant and concurrent validity. Figure 6 shows an example of a VL patient performance.

Fig. 6
figure 6

Example of a maze performed by a VL patient

Full size image

Classical paper-and-pencil tasks

To validate the VSAD, we selected four classical paper-and-pencil tasks that evaluate similar cognitive abilities to those measured by the VSAD. To measure visuospatial working memory, we use the subtest Spatial Memory from the Non-Verbal Weschler Intelligence Scale for Children (Wechsler & Naglieri, 2009), specifically designed for children with language delay or communication disorder such as deafness. To measure mental rotation, we selected the Geometric Puzzles from the NEPSY II (Davis & Thompson, 2011). In this task, the child had to find two geometrical black shapes identical to two models after rotating them mentally in space. To measure selective attention, we used the face cancellation task from the NEPSY (Korkman, Kirk, & Kemp, 1998) where the child had to cancel two types of faces and ignore face distractors. Finally, to measure space orientation, we used the LABY 5-12 (Marquet-Doléac et al., 2010) where the child had to successfully find the exit of different squared and circular mazes. Table 1 summarizes the list of all dependent variables, their abbreviation, and their associated paper-and-pencil task.

Table 1 List of all dependent variables
Full size table

Participants

A total of 67 children participated to the study. Among them, 13 children had VL. This was diagnosed after a medical examination performed by a senior ear, nose, and throat physician on the basis of a videonystagmography with bithermal caloric irrigation (performed for the left and right ears at 44°C and 30°C), cervical vestibular-evoked myogenic potentials (cVEMP) and/or ocular vestibular-evoked myogenic potentials (oVEMP) (when possible). The children were recruited through consultation for dizziness, balance disorders, clumsiness, or other related learning difficulties in the Ear-Nose and Throat department of an academic hospital (**currently anonymized**). The control participants (n = 54) were recruited through posters displayed in schools, in some association groups and through the researcher’s contacts. The groups were matched for mean age (M = 10.5, SD = 3.9 years for vestibular patients and M = 10.8, SD = 3 years for controls; t(71) = 0.201, p = .841).

Some participants did not perform all the tasks due to fatigue or software problems. We noted the exact number of participants relative to each statistical test and ensured that the age between the two groups remained matched. From the control participants, 48 self-reported being right-handed and six left-handed. For the patient group, nine were right-handed, three were left-handed, and one was ambidextrous. Among the 13 children with VL, eight had deafness or hard of hearing, while five have normal hearing.

Patient and control participants had normal or corrected vision, with no spontaneous nystagmus, neurological, psychiatric, or muscular disorders. The study was approved by the hospital ethics committee (see ClinicalTrials.gov Identifier: NCT02533739), and all procedures performed in the study were in accordance with the ethical standards of the institutional and national research committees, and with the 1964 Helsinki Declaration and amendments. The testing session was approximately 1 h, and breaks were provided throughout if necessary. The testing sessions were conducted in the ENT department of the academic hospital, or at the participant’s house.

Statistical analyses

We conducted statistical tests implemented using IBM SPSS Statistics, version 25. Because of the different sample sizes between our groups, and violations of normality for the majority of the dependent variables (verified with the Kolmogorov–Smirnov test), nonparametric analyses were used. We first analyzed concurrent validity by measuring Spearman’s coefficient (r) between the VSAD and traditional paper-and-pencil tasks for the control participants groups. Secondly, to evaluate the performance's stability, we investigated test–retest reliability using intraclass correlation coefficients (ICC) based on mean ratings and absolute agreement parameters with a two-way mixed model (Koo & Li, 2016; Shrout & Fleiss, 1979). Reliability was considered as poor (ICC ≤ 0.40), moderate (0.40 < ICC < 0.75), or excellent (ICC > 0.75) (Andresen, 2000; Fleiss, Levin, & Paik, 1981). Finally, using Mann–Whitney U tests, we analyzed the ability of the VSAD to discriminate between children with VL and healthy control participants (discriminant validity). For this analysis, we only investigated the VSAD tasks that were significantly correlated to traditional pen-and-paper neuropsychological measures. To accommodate the multiple statistics, we adjusted the p values using the false discovery rate method (Benjamini & Hochberg, 1995; Curran-Everett, 2000).

Results

Concurrent validity

Spearman correlation analyses between the VSAD computerized variables and the associated classical paper-and-pencil task measurements showed a strong relationship between the VSAD and the equivalent traditional measures of visuospatial memory and mental rotation (Spearman’s rho between 0.432 and 0.657 all corrected p values < 0.001 for visuospatial memory and < 0.005 for mental rotation, respectively). The VSAD space orientation task compared to traditional measures also showed significant correlations (Spearman’s rho between 0.360 and 0.357, all corrected correlations p < 0.05). However, the selective attention task only showed a significant correlation for the number of errors in the space rockets task and the number of errors in the face’s cancellation task (corrected p < 0.01). Table 2 summarizes the correlation values and their significance between all tasks. As selective attention VSAD tasks were not significantly correlated to the traditional neuropsychological measures of selective attention, we did not perform group comparison analyses for these measures.

Table 2 Concurrent validity and test–retest reliability
Full size table

Test–retest reliability

Thirty participants (VL and controls) performed a second test 1 month following the first test. We excluded the analyses on selective attention performances due to issues with concurrent validity. The test–retest ICC analyses showed moderate to excellent reliability (ICC from 0.389 to 0.913, all corrected p < 0.05). Table 2 provides the details for all tasks.

Discriminant validity

Before correcting for multiple analyses, visuospatial working memory and mental rotation tasks showed significant differences between groups (see Table 3). Children with VL tended to show a total average lower memory score (span) in the forward and backward subtasks (M = 4.77, SD = 1.09, Mdn = 5, and M = 4.15, SD = 1.625, Mdn = 5, respectively) compared to controls (M = 5.35, SD =1.44, Mdn = 5 and M = 5.26, SD = 1.52, Mdn = 4; U = 238, p = .030, r = .33 and U = 212.5, p = .012, r = .29, respectively). Similarly, children with VL tended to show reduced performance in the blazon task evaluating mental rotation abilities (M = 7.85, SD = 2.44) compared to the control group (M = 9.43, SD = 1.792), U = 202, p = .014, r = .36. However, after correcting for multiple testing, differences between groups were no longer significant for any of these tasks. In addition, the performance of the children for the averaged time by correct item in those tasks and their performance for the space orientation task (mazes) were not statistically different between groups.

Table 3 Group comparison analyses
Full size table

Discussion

We present here the VSAD, a new computerized battery implemented in a graphic tablet within the Metrisquare DiagnosIS platform. The VSAD provides different child-friendly subtasks to evaluate visuospatial working memory, mental rotation, selective attention, and space orientation abilities. All these newly developed computerized tasks correlate with traditional neuropsychological paper-and-pencil measures, except for selective attention measures. However, the paper-and-pencil task used in our study (the face cancellation from the NEPSY) could partially explain this lack of reliability, in particular because of the testing procedure. The face cancellation is limited to 180 s (Korkman & Kirk, 2006), providing different results through age with younger children unable to finish the task (and thus making more omissions) and older children, able to finish the task, and making fewer omissions. In our task, we do not stop the task after a certain time. This leads to a ceiling effect where all the children have fewer omissions and errors. This lack of sensitivity prevents us at this stage from drawing any conclusions from our sample of children with VL for the VSAD selective attention tasks but does not prevent determining a cut-off time score in future studies on a larger healthy controls database. Future studies should try to replicate our analyses using another paper-and-pencil selective attentional task. Besides this, all the other tasks newly developed correlates strongly with the traditional neuropsychological paper-and-pencil measures and demonstrates strong concurrent validity. Complementary, the moderate to high test–retest reliability confirmed the performance stability through time of the VSAD.

The ability of the VSAD to discriminate children with VL from healthy controls is nevertheless not yet confirmed with our study. VL in children is a rare condition (Rine, 2009), which makes it difficult to perform studies on large patient samples with sufficient power. This lack of significance after correcting for multiple testing can partially be explained by the large variability of the patient’s performances and the small size of this particularly rare population. This variability could also be linked to the additional deafness present in some of our children with VL. Although our battery has been developed with the goal in mind to be usable for deaf children (due to the written consigns and the possibility to explain through gestures the tasks with the examples provided before testing), our preliminary results on a small population do not allow taking into account the impact of deafness. However, it has been previously shown that deafness can impact attentional processes in several different ways (Bavelier et al., 2000; Bavelier, Dye, & Hauser, 2006; Dye & Bavelier, 2010; Dye, Hauser, & Bavelier, 2009; Hall & Bavelier, 2009; Koo, Crain, LaSasso, & Eden, 2008), although VL is never taken into account in these studies. Our discriminant analysis therefore brings new important information that could be useful for future larger studies. For example, the VSAD provides additional measures such as reaction time that has never been investigated in VL children. While we could expect slower averaged reaction time linked to lower scores, as it has been previously shown in similar adult studies (Brandt et al., 2005; Popp et al., 2017), it is interesting to note that our VL children sample does not seem to show these differences, even before correcting for multiple testing.

Despite this lack of discriminant validity within a sample of children with VL, the VSAD provides new computerized child-friendly reliable measures with good concurrent validity that could be useful for measuring visuospatial abilities in other child populations that experienced similar difficulties such as children with dyspraxia or dyslexia syndrome. We created a complementary automatic scoring algorithm for the selective attention tasks that provides additional dependent variables. These scores could be useful for future research but have not been integrated in this study as it was not the main purpose of our research. For example, it is possible to register the number of omissions and errors in the left/ right side of space versus right/left side of space, up and down, by splitting the screen into four quadrants. It is also possible to register the number of houses cancelled with right or left burning window in order to distinguish allo/egocentric neglect. As for the classical Bell test, we also created a left–right asymmetry cue (total number of omissions on the left side - total number of omissions on the right side) for each cancellation task. A higher positive number indicats more omissions in the left side of space. Additionally, the system also provides time stamps for each cancellation line, allowing clinical interpretation of the child strategy (Figs. 3, 4, and 5). Finally, after each task, we added an adapted child-friendly visual analogue scale to measure the subjective satisfaction of the children's performances (see Fig. 7). Future research is needed to ensure the concurrent validity of the selective attention task and those complementary dependent variables with more sensitive traditional paper-and-pencil task that the one used in our study.

Fig. 7
figure 7

Visual analogue scale to measures children satisfaction of his/her performances after the task. The child had to estimate their satisfaction by choosing a face among five models (from very sad to very happy, automatically coded by the software as one to five)

Full size image

In conclusion, the VSAD provides new computerized child-friendly measures that evaluate visuospatial memory, mental rotation, and space orientation, which correlates with classical neuropsychological paper and pencil, stable through times, and allowing the data collection of accurate new measures through reaction times. On the long term, the additional values of the accurate objective measurements provided by the computerized battery such as the VSAD should allow collecting larger amounts of patient data in a shorter period of time and release clinicians from their usual materials and time constraints during neuropsychological testing.