Abstract
Purpose
Autism Spectrum Disorder (ASD) is diagnosed through observation or interview assessments, which is time-consuming, subjective, and with questionable validity and reliability. Thus, we aimed to evaluate the role of machine learning (ML) with neuroimaging data to provide a reliable classification of ASD.
Methods
A systematic search of PubMed, Scopus, and Embase was conducted to identify relevant publications. Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) was used to assess the studies’ quality. A bivariate random-effects model meta-analysis was employed to evaluate the pooled sensitivity, the pooled specificity, and the diagnostic performance through the hierarchical summary receiver operating characteristic (HSROC) curve of ML with neuroimaging data in classifying ASD. Meta-regression was also performed.
Results
Forty-four studies (5697 ASD and 6013 typically developing individuals [TD] in total) were included in the quantitative analysis. The pooled sensitivity for differentiating ASD from TD individuals was 86.25 95% confidence interval [CI] (81.24, 90.08), while the pooled specificity was 83.31 95% CI (78.12, 87.48) with a combined area under the HSROC (AUC) of 0.889. Higgins I2 (> 90%) and Cochran’s Q (p < 0.0001) suggest a high degree of heterogeneity. In the bivariate model meta-regression, a higher pooled specificity was observed in studies not using a brain atlas (90.91 95% CI [80.67, 96.00], p = 0.032). In addition, a greater pooled sensitivity was seen in studies recruiting both males and females (89.04 95% CI [83.84, 92.72], p = 0.021), and combining imaging modalities (94.12 95% [85.43, 97.76], p = 0.036).
Conclusion
ML with neuroimaging data is an exciting prospect in detecting individuals with ASD but further studies are required to improve its reliability for usage in clinical practice.
Similar content being viewed by others
Data availability
Data is available from the corresponding author upon reasonable request.
Code availability
Code is available from the corresponding author upon reasonable request.
References
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. Arlington, VA: American Psychiatric Association 2013
Newschaffer CJ, Curran LK (2003) Autism: an emerging public health problem. Public Health Rep 118:393–399
Ozonoff S, Iosif A-M, Baguio F, Cook IC, Hill MM, Hutman T, Rogers SJ, Rozga A, Sangha S, Sigman M, Steinfeld MB, Young GS (2010) A prospective study of the emergence of early behavioral signs of autism. J Am Acad Child Adolesc Psychiatry 49:256–66.e2
Zwaigenbaum L, Bryson S, Rogers T, Roberts W, Brian J, Szatmari P (2005) Behavioral manifestations of autism in the first year of life. Int J Dev Neurosci 23:143–152
Mandell DS (2005) Factors associated with age of diagnosis among children with autism spectrum disorders. Pediatrics 116:1480–1486
Stahmer AC, Collings NM, Palinkas LA (2005) Early intervention practices for children with autism: descriptions from community providers. Focus on autism and other developmental disabilities 20:66–79
Falkmer T, Anderson K, Falkmer M, Horlin C (2013) Diagnostic procedures in autism spectrum disorders: a systematic literature review. Eur Child Adolesc Psychiatry 22:329–340
Randall M, Albein-Urios N, Brignell A, Gulenc A, Hennel S, Coates C, Symeonides C, Hiscock H, Marraffa C, Silove N, Bayl V, Woolfenden S, Williams K (2016) Diagnosing autism: Australian paediatric research network surveys: diagnosing autism: an APRN survey. J Paediatr Child Health 52:11–17
Taylor LJ, Eapen V, Maybery M, Midford S, Paynter J, Quarmby L, Smith T, Williams K, Whitehouse AJO (2017) Brief report: an exploratory study of the diagnostic reliability for autism spectrum disorder. J Autism Dev Disord 47:1551–1558
Dosreis S, Weiner CL, Johnson L, Newschaffer CJ (2006) Autism spectrum disorder screening and management practices among general pediatric providers. J Dev Behav Pediatr 27:S88–S94
Antezana L, Scarpa A, Valdespino A, Albright J, Richey JA (2017) Rural trends in diagnosis and services for autism spectrum disorder. Front Psychol 8:590
Piven J, Elison J, Zylka M (2017) Toward a conceptual framework for early brain and behavior development in autism. Mol Psychiatry 22:1385–1394
Di X, Azeez A, Li X, Haque E, Biswal BB (2018) Disrupted focal white matter integrity in autism spectrum disorder: a voxel-based meta-analysis of diffusion tensor imaging studies. Prog Neuropsychopharmacol Biol Psychiatry 82:242–248
Tyszka JM, Kennedy DP, Paul LK, Adolphs R (2014) Largely typical patterns of resting-state functional connectivity in high-functioning adults with autism. Cereb Cortex 24:1894–1905
Kapur S, Phillips AG, Insel TR (2012) Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Mol Psychiatry 17:1174–1179
McAlonan GM. Mapping the brain in autism. A voxel-based MRI study of volumetric differences and intercorrelations in autism. Brain. 2005;128:268–276
Vabalas A, Gowen E, Poliakoff E, Casson AJ (2019) Machine learning algorithm validation with a limited sample size. PloS one 14:e0224365-e
Lanka P, Rangaprakash D, Dretsch MN, Katz JS, Denney TS, Deshpande G (2020) Supervised machine learning for diagnostic classification from large-scale neuroimaging datasets. Brain Imaging Behav 14:2378–2416
Noorbakhsh-Sabet N, Zand R, Zhang Y, Abedi V (2019) Artificial intelligence transforms the future of health care. Am J Med 132:795–801
Jordan MI, Mitchell TM (2015) Machine learning: Trends, perspectives, and prospects. Science 349:255–260
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. BMJ 339:b2700
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. BMJ. 2009;339:b2700
Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MMG, Sterne JAC, Bossuyt PMM (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155:529–536
Lee J, Kim KW, Choi SH, Huh J, Park SH (2015) Systematic review and meta-analysis of studies evaluating diagnostic test accuracy: a practical review for clinical researchers-part II. Statistical Methods of Meta-Analysis. Korean journal of radiology 16:1175–1187
Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH (2005) Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 58:982–990
Riley RD, Higgins JPT, Deeks JJ (2011) Interpretation of random effects meta-analyses. BMJ 342:d549
Emerson RW, Adams C, Nishino T, Hazlett HC, Wolff JJ, Zwaigenbaum L, Constantino JN, Shen MD, Swanson MR, Elison JT, Kandala S, Estes AM, Botteron KN, Collins L, Dager SR, Evans AC, Gerig G, Gu H, McKinstry RC, Paterson S, Schultz RT, Styner M, Schlaggar BL, Pruett JR, Piven J (2017) Functional neuroimaging of high-risk 6-month-old infants predicts a diagnosis of autism at 24 months of age. Science Translational Medicine 9:eaag2882
Shen MD, Nordahl CW, Li DD, Lee A, Angkustsiri K, Emerson RW, Rogers SJ, Ozonoff S, Amaral DG (2018) Extra-axial cerebrospinal fluid in high-risk and normal-risk children with autism aged 2–4 years: a case-control study. The Lancet Psychiatry 5:895–904
Xiao X, Fang H, Wu J, Xiao C, Xiao T, Qian L, Liang F, Xiao Z, Chu KK, Ke X (2017) Diagnostic model generated by MRI-derived brain features in toddlers with autism spectrum disorder. Autism Res 10:620–630
Abraham A, Milham MP, Di Martino A, Craddock RC, Samaras D, Thirion B, Varoquaux G (2017) Deriving reproducible biomarkers from multi-site resting-state data: an Autism-based example. Neuroimage 147:736–745
Akhavan Aghdam M, Sharifi A, Pedram MM (2018) Combination of rs-fMRI and sMRI data to discriminate autism spectrum disorders in young children using deep belief network. J Digit Imaging 31:895–903
Bajestani GS, Behrooz M, Khani AG, Nouri-Baygi M, Mollaei A (2019) Diagnosis of autism spectrum disorder based on complex network features. Comput Methods Programs Biomed 177:277–283
Bosl WJ, Loddenkemper T, Nelson CA (2017) Nonlinear EEG biomarker profiles for autism and absence epilepsy. Neuropsychiatric Electrophysiology 3
Brahim A, Farrugia N (2020) Graph fourier transform of fMRI temporal signals based on an averaged structural connectome for the classification of neuroimaging. Artificial Intelligence in Medicine 106:101870
Chen T, Chen Y, Yuan M, Gerstein M, Li T, Liang H, Froehlich T, Lu L (2020) The development of a practical artificial intelligence tool for diagnosing and evaluating autism spectrum disorder: multicenter study. JMIR Medical Informatics 8:e15767
Dekhil O, Hajjdiab H, Shalaby A, Ali MT, Ayinde B, Switala A, Elshamekh A, Ghazal M, Keynton R, Barnes G, El-Baz A (2018) Using resting state functional MRI to build a personalized autism diagnosis system. PloS One 13:e0206351
Duchesnay E, Cachia A, Boddaert N, Chabane N, Mangin J-F, Martinot J-L, Brunelle F, Zilbovicius M (2011) Feature selection and classification of imbalanced datasets: application to PET images of children with autistic spectrum disorders. Neuroimage 57:1003–1014
Eill A, Jahedi A, Gao Y, Kohli JS, Fong CH, Solders S, Carper RA, Valafar F, Bailey BA, Müller R-A (2019) Functional connectivities are more informative than anatomical variables in diagnostic classification of autism. Brain connectivity 9:604–612
Gori I, Giuliano A, Muratori F, Saviozzi I, Oliva P, Tancredi R, Cosenza A, Tosetti M, Calderoni S, Retico A (2015) Gray matter alterations in young children with autism spectrum disorders: comparing morphometry at the voxel and regional level: gray matter alterations in ASD children. J Neuroimaging 25:866–874
Grossi E, Olivieri C, Buscema M (2017) Diagnosis of autism through EEG processed by advanced computational algorithms: a pilot study. Comput Methods Programs Biomed 142:73–79
Grossi E, Buscema M, Della Torre F, Swatzyna RJ (2019) The “MS-ROM/IFAST” model, a novel parallel nonlinear EEG analysis technique, distinguishes ASD subjects from children affected with other neuropsychiatric disorders with high degree of accuracy. Clin EEG Neurosci 50:319–331
Iidaka T (2015) Resting state functional magnetic resonance imaging and neural network classified autism and control. Cortex 63:55–67
Ingalhalikar M, Parker D, Bloy L, Roberts TPL, Verma R (2011) Diffusion based abnormality markers of pathology: toward learned diagnostic prediction of ASD. Neuroimage 57:918–927
Irimia A, Lei X, Torgerson CM, Jacokes ZJ, Abe S, Van Horn JD (2018) Support vector machines, multidimensional scaling and magnetic resonance imaging reveal structural brain abnormalities associated with the interaction between autism spectrum disorder and sex. Front Comput Neurosci 12:93
Jiao Y, Chen R, Ke X, Chu K, Lu Z, Herskovits EH (2010) Predictive models of autism spectrum disorder based on brain regional cortical thickness. Neuroimage 50:589–599
Kam T-E, Suk H-I, Lee S-W (2017) Multiple functional networks modeling for autism spectrum disorder diagnosis: multiple functional networks modeling for ASD diagnosis. Hum Brain Mapp 38:5804–5821
Payabvash S, Palacios EM, Owen JP, Wang MB, Tavassoli T, Gerdes M, Brandes-Aitken A, Cuneo D, Marco EJ, Mukherjee P (2019) White matter connectome edge density in children with autism spectrum disorders: potential imaging biomarkers using machine-learning models. Brain connectivity 9:209–220
Pham T-H, Vicnesh J, Wei JKE, Oh SL, Arunkumar N, Abdulhay EW, Ciaccio EJ, Acharya UR (2020) Autism spectrum disorder diagnostic system using HOS bispectrum with EEG signals. Int J Environ Res Public Health 17:971
Schirmer MD, Venkataraman A, Rekik I, Kim M, Mostofsky SH, Nebel MB, Rosch K, Seymour K, Crocetti D, Irzan H, Hütel M, Ourselin S, Marlow N, Melbourne A, Levchenko E, Zhou S, Kunda M, Lu H, Dvornek NC, Zhuang J, Pinto G, Samal S, Zhang J, Bernal-Rusiel JL, Pienaar R, Chung AW (2021) Neuropsychiatric disease classification using functional connectomics - results of the connectomics in neuroimaging transfer learning challenge. Medical Image Analysis 70:101972
Spera G, Retico A, Bosco P, Ferrari E, Palumbo L, Oliva P, Muratori F, Calderoni S (2019) Evaluation of altered functional connections in male children with autism spectrum disorders on multiple-site data optimized with machine learning. Front Psych 10:620
Xiao Z, Wu J, Wang C, Jia N, Yang X (2019) Computer-aided diagnosis of school-aged children with ASD using full frequency bands and enhanced SAE: a multi-institution study. Exp Ther Med 17:4055–4063
Zhang F, Savadjiev P, Cai W, Song Y, Rathi Y, Tunç B, Parker D, Kapur T, Schultz RT, Makris N, Verma R, O’Donnell LJ (2018) Whole brain white matter connectivity analysis using machine learning: an application to autism. Neuroimage 172:826–837
Ecker C, Marquand A, Mourão-Miranda J, Johnston P, Daly EM, Brammer MJ, Maltezos S, Murphy CM, Robertson D, Williams SC, Murphy DGM (2010) Describing the brain in autism in five dimensions—magnetic resonance imaging-assisted diagnosis of autism spectrum disorder using a multiparameter classification approach. J Neurosci 30:10612–10623
Ecker C, Rocha-Rego V, Johnston P, Mourao-Miranda J, Marquand A, Daly EM, Brammer MJ, Murphy C, Murphy DG (2010) Investigating the predictive value of whole-brain structural MR scans in autism: a pattern classification approach. Neuroimage 49:44–56
Yahata N, Morimoto J, Hashimoto R, Lisi G, Shibata K, Kawakubo Y, Kuwabara H, Kuroda M, Yamada T, Megumi F, Imamizu H, Náñez JE Sr, Takahashi H, Okamoto Y, Kasai K, Kato N, Sasaki Y, Watanabe T, Kawato M (2016) A small number of abnormal brain connections predicts adult autism spectrum disorder. Nat Commun 7:11254
Yassin W, Nakatani H, Zhu Y, Kojima M, Owada K, Kuwabara H, Gonoi W, Aoki Y, Takao H, Natsubori T, Iwashiro N, Kasai K, Kano Y, Abe O, Yamasue H, Koike S (2020) Machine-learning classification using neuroimaging data in schizophrenia, autism, ultra-high risk and first-episode psychosis. Transl Psychiatry 10:278
Chen CP, Keown CL, Jahedi A, Nair A, Pflieger ME, Bailey BA, Müller R-A (2015) Diagnostic classification of intrinsic functional connectivity highlights somatosensory, default mode, and visual regions in autism. NeuroImage clinical 8:238–245
Deshpande G, Libero LE, Sreenivasan KR, Deshpande HD, Kana RK (2013) Identification of neural connectivity signatures of autism using machine learning. Front Hum Neurosci 7:670
Eslami T, Mirjalili V, Fong A, Laird AR, Saeed F (2019) ASD-DiagNet: A hybrid learning approach for detection of autism spectrum disorder using fMRI data. Front Neuroinform 13:70
Fu Y, Zhang J, Li Y, Shi J, Zou Y, Guo H, Li Y, Yao Z, Wang Y, Hu B (2021) A novel pipeline leveraging surface-based features of small subcortical structures to classify individuals with autism spectrum disorder. Progress in Neuro-Psychopharmacology & Biological Psychiatry 104:109989
Ghiassian S, Greiner R, Jin P, Brown MRG (2016) Using functional or structural magnetic resonance images and personal characteristic data to identify ADHD and autism. PloS One 11:e0166934
Heinsfeld AS, Franco AR, Craddock RC, Buchweitz A, Meneguzzi F (2018) Identification of autism spectrum disorder using deep learning and the ABIDE dataset. NeuroImage clinical 17:16–23
Huang F, Tan E-L, Yang P, Huang S, Ou-Yang L, Cao J, Wang T, Lei B (2020) Self-weighted adaptive structure learning for ASD diagnosis via multi-template multi-center representation. Medical Image Analysis 63:101662
Kassraian-Fard P, Matthis C, Balsters JH, Maathuis MH, Wenderoth N (2016) Promises, pitfalls, and basic guidelines for applying machine learning classifiers to psychiatric imaging data, with autism as an example. Front Psych 7:177
Kazeminejad A, Sotero RC (2019) Topological properties of resting-state fMRI functional networks improve machine learning-based autism classification. Front Neurosci 12:1018
Li H, Parikh NA, He L (2018) A novel transfer learning approach to enhance deep neural network classification of brain functional connectomes. Front Neurosci 12:491
Plitt M, Barnes KA, Martin A (2014) Functional connectivity classification of autism identifies highly predictive brain features but falls short of biomarker standards. NeuroImage clinical 7:359–366
Rakić M, Cabezas M, Kushibar K, Oliver A, Lladó X (2020) Improving the detection of autism spectrum disorder by combining structural and functional MRI information. NeuroImage Clinical 25:102181
Tomasiello S (2020) A granular functional network classifier for brain diseases analysis. Comput Methods Biomech Biomed Engin 8:382–388
Zu C, Gao Y, Munsell B, Kim M, Peng Z, Cohen JR, Zhang D, Wu G (2019) Identifying disease-related subnetwork connectome biomarkers by sparse hypergraph learning. Brain Imaging Behav 13:879–892
Fombonne E (2009) Epidemiology of pervasive developmental disorders. Pediatr Res 65:591–598
Halladay AK, Bishop S, Constantino JN, Daniels AM, Koenig K, Palmer K, Messinger D, Pelphrey K, Sanders SJ, Singer AT, Taylor JL, Szatmari P (2015) Sex and gender differences in autism spectrum disorder: summarizing evidence gaps and identifying emerging areas of priority. Molecular autism 6:36
Carter AS, Black DO, Tewani S, Connolly CE, Kadlec MB, Tager-Flusberg H (2007) Sex differences in toddlers with autism spectrum disorders. J Autism Dev Disord 37:86–97
Ros-Demarize R, Bradley C, Kanne SM, Warren Z, Boan A, Lajonchere C, Park J, Carpenter LA (2020) ASD symptoms in toddlers and preschoolers: an examination of sex differences. Autism Res 13:157–166
Stigler KA, McDonald BC, Anand A, Saykin AJ, McDougle CJ (2011) Structural and functional magnetic resonance imaging of autism spectrum disorders. Brain Res 1380:146–161
Barttfeld P, Wicker B, Cukier S, Navarta S, Lew S, Sigman M (2011) A big-world network in ASD: dynamical connectivity analysis reflects a deficit in long-range connections and an excess of short-range connections. Neuropsychologia 49:254–263
Koprowski R, Foster KR (2018) Machine learning and medicine: book review and commentary. Biomed Eng Online 17:17
Uddin S, Khan A, Hossain ME, Moni MA (2019) Comparing different supervised machine learning algorithms for disease prediction. BMC Med Inform Decis Mak 19:281
Dickie DA, Shenkin SD, Anblagan D, Lee J, BlesaCabez M, Rodriguez D, Boardman JP, Waldman A, Job DE, Wardlaw JM (2017) Whole brain magnetic resonance image atlases: a systematic review of existing atlases and caveats for use in population imaging. Front Neuroinform 11:1
Jo T, Nho K, Saykin AJ (2019) Deep learning in Alzheimer’s disease: diagnostic classification and prognostic prediction using neuroimaging data. Frontiers in aging neuroscience 11:220
Kambeitz J, Kambeitz-Ilankovic L, Leucht S, Wood S, Davatzikos C, Malchow B, Falkai P, Koutsouleris N (2015) Detecting neuroimaging biomarkers for schizophrenia: a meta-analysis of multivariate pattern recognition studies. Neuropsychopharmacology 40:1742–1751
Librenza-Garcia D, Kotzian BJ, Yang J, Mwangi B, Cao B, Pereira Lima LN, Bermudez MB, Boeira MV, Kapczinski F, Passos IC (2017) The impact of machine learning techniques in the study of bipolar disorder: a systematic review. Neurosci Biobehav Rev 80:538–554
Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D (2019) Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine 17:195
Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E (2014) Reducing waste from incomplete or unusable reports of biomedical research. The Lancet 383:267–276
Collins GS, Moons KGM (2012) Comparing risk prediction models. BMJ 344:e3186
Vollmer S, Mateen BA, Bohner G, Kiraly FJ, Ghani R, Jonsson P et al (2020) Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ 368:l6927
Funding
This study was funded by the National Institute for Health Research UCLH Biomedical Research Centre (BRC399/NS/RB/101410).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval and consent to participate
This article does not contain any studies with human participants performed by any of the authors.
Consent for publication
Not applicable.
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Da-Yea Song and Constantin-Cristian Topriceanu share co-first authorship.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Song, DY., Topriceanu, CC., Ilie-Ablachim, D.C. et al. Machine learning with neuroimaging data to identify autism spectrum disorder: a systematic review and meta-analysis. Neuroradiology 63, 2057–2072 (2021). https://doi.org/10.1007/s00234-021-02774-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00234-021-02774-z