Skip to main content

Advertisement

SpringerLink
Log in

Automated Detection Approaches to Autism Spectrum Disorder Based on Human Activity Analysis: A Review

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Autism Spectrum Disorder (ASD) is a neuro-developmental disorder that limits social and cognitive abilities. ASD has no cure so early diagnosis is important for reducing its impact. The current behavioral observation-based subjective-diagnosis systems (e.g., DSM-5 or ICD-10) frequently misdiagnose subjects. Therefore, researchers are attempting to develop automated diagnosis systems with minimal human intervention, quicker screening time, and better outreach. This paper is a PRISMA-based systematic review examining the potential of automated autism detection system with Human Activity Analysis (HAA) to look for distinctive ASD characteristics such as repetitive behavior, abnormal gait and visual saliency. The literature from 2011 onward is qualitatively and quantitatively analyzed to investigate whether HAA can identify the features of ASD, the level of its classification accuracy, the degree of human intervention, and screening time. Based on these findings, we discuss the approaches, challenges, resources, and future directions in this area. According to our quantitative assessment of the dataset Zunino et al. (IEEE: 3421–3426, 2018 [1]), Inception v3 and LSTM Zunino et al. (IEEE: 3421–3426, 2018 [1]) give the highest accuracy (89%) for repetitive behavior. For abnormal gait-based approach, the multilayer perceptron gives 98% accuracy based on 18 features from dataset Abdulrahman et al. (COMPUSOFT: An International Journal of Advanced Computer Technology 9(8):3791–3797, 2020 [2]). For gaze pattern, a saliency-metric feature-based learning Rahman et al. (Int Conf Pattern Recognit, 2020 [3]) gives 99% accuracy on dataset Duan et al. (Proceedings of the 10th ACM Multimedia Systems Conference: 255–260, 2019 [4]), while an algorithm involving statistical features and Decision Trees yields an accuracy of 76% on dataset Yaneva et al. (Proceedings of the Internet of Accessible Things. W4A ’18, Association for Computing Machinery, New York, NY, USA, 1–10, 2018 [5]). In terms of the state of the art, fully automated HAA systems for ASD diagnosis show promise but are still in developmental stages. However, this is an active research field, and HAA has good prospects for helping to diagnose ASD objectively in less time with better accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Zunino A, Morerio P, Cavallo A, Ansuini C, Podda J, Battaglia F, etal. Video gesture analysis for autism spectrum disorder detection. In: 2018 24th International Conference on Pattern Recognition (ICPR). IEEE; 2018. p. 3421–3426.

  2. Abdulrahman A, Hadi I, Rajihy Y. Generating 3D dataset of Gait and Full body movement of children with Autism spectrum disorders collected by Kinect v2 camera. COMPUSOFT: An International Journal of Advanced Computer Technology. 2020;9(8):3791–3797.

  3. Rahman S, Rahman S, Shahid O, Abdullah M, Sourov JA. Classifying Eye-Tracking Data Using Saliency Maps. Int Conf Pattern Recognit. 2020.

  4. Duan H, Zhai G, Min X, Che Z, Fang Y, Yang X, et al. A dataset of eye movements for the children with autism spectrum disorder. In: Proceedings of the 10th ACM Multimedia Systems Conference; 2019. p. 255–260.

  5. Yaneva V, Ha LA, Eraslan S, Yesilada Y, Mitkov R. Detecting Autism Based on Eye-Tracking Data from Web Searching Tasks. In: Proceedings of the Internet of Accessible Things. W4A ’18. New York, NY, USA: Association for Computing Machinery; 2018. p. 1–10.

  6. World H Organization. Autism spectrum disorders. World Health Organization. 2019 Nov. Available from: https://www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders.

  7. Hossain MD, Ahmed HU, Uddin MJ, Chowdhury WA, Iqbal MS, Kabir RI, et al. Autism Spectrum disorders (ASD) in South Asia: a systematic review. BMC Psychiatry. 2017;17(1):1–7.

    Article  Google Scholar 

  8. Hamdoun O. Autism Spectrum Disorders, is it Under Reported In Third World Countries. Am J Biomed Sci Res. 2019;4(4):292–3.

    Article  Google Scholar 

  9. Meilleur AAS, Jelenic P, Mottron L. Prevalence of clinically and empirically defined talents and strengths in autism. J Autism Dev Disord. 2015;45(5):1354–67.

    Article  Google Scholar 

  10. Fakhoury M. Autistic spectrum disorders: A review of clinical features, theories and diagnosis. Int J Dev Neurosci. 2015;43:70–7.

    Article  Google Scholar 

  11. Jiang Yh, Yuen RK, Jin X, Wang M, Chen N, Wu X, et al. Detection of clinically relevant genetic variants in autism spectrum disorder by whole-genome sequencing. Am J Human Gen. 2013;93(2):249–63.

    Article  Google Scholar 

  12. Zwaigenbaum L, Bauman ML, Choueiri R, Fein D, Kasari C, Pierce K, et al. Early Identification and Interventions for Autism Spectrum Disorder: Executive Summary. Pediatrics. 2015;136(Supplement):S1–9.

    Article  Google Scholar 

  13. Volkmar FR, Reichow B, McPartland J. Classification of autism and related conditions: progress, challenges, and opportunities. Dialogues Clin Neurosci. 2012;14(3):229.

    Article  Google Scholar 

  14. Hoefman R, Payakachat N, van Exel J, Kuhlthau K, Kovacs E, Pyne J, et al. Caring for a child with autism spectrum disorder and parents quality of life: application of the CarerQol. J Autism Dev Dis. 2014;44(8):1933–45.

    Google Scholar 

  15. Lord C, Risi S, DiLavore PS, Shulman C, Thurm A, Pickles A. Autism from 2 to 9 years of age. Arch Gen Psychiatry. 2006;63(6):694–701.

    Article  Google Scholar 

  16. Association AP. Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Pub. 2013.

  17. Maenner MJ, Rice CE, Arneson CL, Cunniff C, Schieve LA, Carpenter LA, et al. Potential impact of DSM-5 criteria on autism spectrum disorder prevalence estimates. JAMA Psychiat. 2014;71(3):292–300.

    Article  Google Scholar 

  18. Bryson SE, Rogers SJ, Fombonne E. Autism spectrum disorders: early detection, intervention, education, and psychopharmacological management. Canadian J Psych. 2003;48(8):506–16.

    Article  Google Scholar 

  19. Kleinman JM, Robins DL, Ventola PE, Pandey J, Boorstein HC, Esser EL, et al. The modified checklist for autism in toddlers: a follow-up study investigating the early detection of autism spectrum disorders. J Autism Dev Disord. 2008;38(5):827–39.

    Article  Google Scholar 

  20. Hazlett HC, Gu H, Munsell BC, Kim SH, Styner M, Wolff JJ, et al. Early brain development in infants at high risk for autism spectrum disorder. Nature. 2017;542(7641):348–51.

    Article  Google Scholar 

  21. Tager-Flusberg H. Brain Imaging Studies in Autism Spectrum Disorders. The Asperger / Autism Network (AANE). 2017 Feb. Available from: https://www.aane.org/brain-imaging-studies-autism-spectrum-disorders/.

  22. Zhang S, Wei Z, Nie J, Huang L, Wang S, Li Z. A review on human activity recognition using vision-based method. J Healthcare Eng. 2017.

  23. Srivastava AK, Biswas K, Tripathi V. A Robust Framework for Effective Human Activity Analysis. In: International Conference on Innovative Computing and Communications. Springer; 2019:331–337.

  24. Wu D, Sharma N, Blumenstein M. International joint conference on neural networks (IJCNN). IEEE. Recent advances in video based human action recognition using deep learning: a review. 2017;2017:2865–72.

  25. Aly S, Trubanova A, Abbott L, White S, Youssef A. VT-KFER: A Kinect-based RGBD+ time dataset for spontaneous and non-spontaneous facial expression recognition. In: 2015 International Conference on Biometrics (ICB). IEEE; 2015:90–97.

  26. Faso DJ, Sasson NJ, Pinkham AE. Evaluating posed and evoked facial expressions of emotion from adults with autism spectrum disorder. J Autism Dev Disord. 2015;45(1):75–89.

    Article  Google Scholar 

  27. Harms MB, Martin A, Wallace GL. Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychol Rev. 2010;20(3):290–322.

    Article  Google Scholar 

  28. Rehg JM, Rozga A, Abowd GD, Goodwin MS. Behavioral imaging and autism. IEEE Pervasive Comput. 2014;13(2):84–7.

    Article  Google Scholar 

  29. Militerni R, Bravaccio C, Falco C, Fico C, Palermo MT. Repetitive behaviors in autistic disorder. Euro Child Adol Psych. 2002;11(5):210–8.

    Article  Google Scholar 

  30. Manicolo O, Brotzmann M, Hagmann-von Arx P, Grob A, Weber P. Gait in children with infantile/atypical autism: Age-dependent decrease in gait variability and associations with motor skills. Eur J Paediatr Neurol. 2019;23(1):117–25.

    Article  Google Scholar 

  31. Wang S, Jiang M, Duchesne XM, Laugeson EA, Kennedy DP, Adolphs R, et al. Atypical visual saliency in autism spectrum disorder quantified through model-based eye tracking. Neuron. 2015;88(3):604–16.

    Article  Google Scholar 

  32. Chawarska K, Shic F. Looking but not seeing: Atypical visual scanning and recognition of faces in 2 and 4-year-old children with autism spectrum disorder. J Autism Dev Disord. 2009;39(12):1663.

    Article  Google Scholar 

  33. Subbaraju V, Suresh MB, Sundaram S, Narasimhan S. Identifying differences in brain activities and an accurate detection of autism spectrum disorder using resting state functional-magnetic resonance imaging: A spatial filtering approach. Med Image Anal. 2017;35:375–89.

    Article  Google Scholar 

  34. Eslami T, Mirjalili V, Fong A, Laird AR, Saeed F. ASD-DiagNet: A Hybrid Learning Approach for Detection of Autism Spectrum Disorder Using fMRI Data. Frontiers in Neuroinformatics. 2019 Nov;13.

  35. Sherkatghanad Z, Akhondzadeh M, Salari S, Zomorodi-Moghadam M, Abdar M, Acharya UR, et al. Automated Detection of Autism Spectrum Disorder Using a Convolutional Neural Network. Frontiers in Neuroscience. 2020 Jan; 13.

  36. Kandalaft MR, Didehbani N, Krawczyk DC, Allen TT, Chapman SB. Virtual reality social cognition training for young adults with high-functioning autism. J Autism Dev Disord. 2013;43(1):34–44.

    Article  Google Scholar 

  37. Welch KC, Lahiri U, Liu C, Weller R, Sarkar N, Warren Z. An affect-sensitive social interaction paradigm utilizing virtual reality environments for autism intervention. In: International conference on human-computer interaction. Springer; 2009. p. 703–712.

  38. Zhang L, Wade JW, Bian D, Swanson A, Warren Z, Sarkar N. Data fusion for difficulty adjustment in an adaptive virtual reality game system for autism intervention. In: International Conference on Human-Computer Interaction. Springer; 2014. p. 648–652.

  39. Lahiri U, Warren Z, Sarkar N. Dynamic gaze measurement with adaptive response technology in Virtual Reality based social communication for autism. In: 2011 International Conference on Virtual Rehabilitation. IEEE; 2011. p. 1–8.

  40. Lahiri U, Warren Z, Sarkar N. Design of a gaze-sensitive virtual social interactive system for children with autism. IEEE Trans Neural Syst Rehabil Eng. 2011;19(4):443–52.

    Article  Google Scholar 

  41. Bekele E, Wade J, Bian D, Fan J, Swanson A, Warren Z. In: 2016 IEEE Virtual Reality (VR). IEEE. Multimodal adaptive social interaction in virtual environment (MASI-VR) for children with Autism spectrum disorders (ASD). 2016;2016:121–30.

    Google Scholar 

  42. Hyde KK, Novack MN, LaHaye N, Parlett-Pelleriti C, Anden R, Dixon DR, et al. Applications of supervised machine learning in autism spectrum disorder research: a review. Rev J Autism Dev Disord. 2019;6(2):128–46.

    Article  Google Scholar 

  43. Thabtah F. Machine learning in autistic spectrum disorder behavioral research: A review and ways forward. Inform Health Soc Care. 2019;44(3):278–97.

    Article  Google Scholar 

  44. Song DY, Kim SY, Bong G, Kim JM, Yoo HJ. The Use of Artificial Intelligence in Screening and Diagnosis of Autism Spectrum Disorder: A Literature Review. Journal of the Korean Academy of Child and Adolescent Psychiatry. 2019;30(4):145–52.

    Article  Google Scholar 

  45. Boucenna S, Narzisi A, Tilmont E, Muratori F, Pioggia G, Cohen D, et al. Interactive technologies for autistic children: A review. Cogn Comput. 2014;6(4):722–40.

    Article  Google Scholar 

  46. Sevin JA, Rieske RD, Matson JL. A review of behavioral strategies and support considerations for assisting persons with difficulties transitioning from activity to activity. Rev J Autism Dev Disord. 2015;2(4):329–42.

    Article  Google Scholar 

  47. Reinders NJ, Branco A, Wright K, Fletcher PC, Bryden PJ. Scoping review: physical activity and social functioning in young people with autism spectrum disorder. Front Psychol. 2019;10:120.

    Article  Google Scholar 

  48. Scharoun SM, Wright KT, Robertson-Wilson JE, Fletcher PC, Bryden PJ. Physical activity in individuals with autism spectrum disorders (ASD): a review. Autism-paradigms, recent research and clinical applications. 2017.

  49. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1–34.

    Article  Google Scholar 

  50. Rihawi O, Merad D, Damoiseaux JL. 3D-AD: 3D-autism dataset for repetitive behaviours with kinect sensor. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE; 2017. p. 1–6.

  51. Shihab AI, Dawood FA, Kashmar AH. Data Analysis and Classification of Autism Spectrum Disorder Using Principal Component Analysis. Adv Bioinform. 2020;2020:1–8.

    Article  Google Scholar 

  52. Weinland D, Ronfard R, Boyer E. A survey of vision-based methods for action representation, segmentation and recognition. Comput Vis Image Underst. 2011;115(2):224–41.

    Article  Google Scholar 

  53. Jazouli M, Elhoufi S, Majda A, Zarghili A, Aalouane R. Stereotypical motor movement recognition using microsoft kinect with artificial neural network. World Acad Sci Eng Technol Int J Comput Electr Autom Control Inf Eng. 2016;10(7):1270–4.

    Google Scholar 

  54. Rohrbach A, Rohrbach M, Tandon N, Schiele B. A dataset for movie description. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2015:3202–3212.

  55. Reddy KK, Shah M. Recognizing 50 human action categories of web videos. Mach Vis Appl. 2013;24(5):971–81.

    Article  Google Scholar 

  56. Ryoo MS, Matthies L. First-person activity recognition: What are they doing to me? In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2013:2730–2737.

  57. Chen L, Hoey J, Nugent CD, Cook DJ, Yu Z. Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). 2012;42(6):790–808.

  58. Liu Y, Nie L, Liu L, Rosenblum DS. From action to activity: sensor-based activity recognition. Neurocomputing. 2016;181:108–15.

    Article  Google Scholar 

  59. Duchesnay E, Cachia A, Boddaert N, Chabane N, Mangin JF, Martinot JL, et al. Feature selection and classification of imbalanced datasets. Neuroimage. 2011;57(3):1003–14.

    Article  Google Scholar 

  60. Papagiannopoulou EA, Chitty KM, Hermens DF, Hickie IB, Lagopoulos J. A systematic review and meta-analysis of eye-tracking studies in children with autism spectrum disorders. Soc Neurosci. 2014;9(6):610–32.

    Google Scholar 

  61. Feil-Seifer D, Matarić MJ. In: 2011 6th ACM/IEEE international conference on human-robot interaction (HRI). IEEE. Automated detection and classification of positive vs negative robot interactions with children with autism using distance-based features. 2011;2011:323–30.

    Google Scholar 

  62. Bodfish JW. Stereotypy, self-injury, and related abnormal repetitive behaviors. In: Handbook of intellectual and developmental disabilities. Springer; 2007:481–505.

  63. Leekam S, Tandos J, McConachie H, Meins E, Parkinson K, Wright C, et al. Repetitive behaviours in typically developing 2-year-olds. J Child Psychol Psychiatry. 2007;48(11):1131–8.

    Article  Google Scholar 

  64. Arnott B, McConachie H, Meins E, Fernyhough C, Le Couteur A, Turner M, et al. The frequency of restricted and repetitive behaviors in a community sample of 15-month-old infants. Journal of Developmental & Behavioral Pediatrics. 2010;31(3):223–9.

    Article  Google Scholar 

  65. Richler J, Huerta M, Bishop SL, Lord C. Developmental trajectories of restricted and repetitive behaviors and interests in children with autism spectrum disorders. Dev Psychopathol. 2010;22(1):55.

    Article  Google Scholar 

  66. Goodwin MS, Intille SS, Albinali F, Velicer WF. Automated detection of stereotypical motor movements. J Autism Dev Disord. 2011;41(6):770–82.

    Article  Google Scholar 

  67. Gonçalves N, Rodrigues JL, Costa S, Soares F. In: 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. IEEE. Automatic detection of stereotyped hand flapping movements: two different approaches. 2012;2012:392–7.

    Google Scholar 

  68. Zhao Z, Zhang X, Li W, Hu X, Qu X, Cao X, et al. Applying Machine Learning to Identify Autism With Restricted Kinematic Features. IEEE Access. 2019;7:157614–22.

    Article  Google Scholar 

  69. Jazouli M, Majda A, Merad D, Aalouane R, Zarghili A. Automatic detection of stereotyped movements in autistic children using the Kinect sensor. Int J Biomed Eng Technol. 2019;29(3):201–20.

    Article  Google Scholar 

  70. Coronato A, De Pietro G, Paragliola G. A situation-aware system for the detection of motion disorders of patients with autism spectrum disorders. Expert Syst Appl. 2014;41(17):7868–77.

    Article  Google Scholar 

  71. Sadouk L, Gadi T, Essoufi EH. A novel deep learning approach for recognizing stereotypical motor movements within and across subjects on the autism spectrum disorder. Computational intelligence and neuroscience. 2018.

  72. Jaiswal S, Valstar MF, Gillott A, Daley D. Automatic detection of ADHD and ASD from expressive behaviour in RGBD data. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE; 2017:762–769.

  73. Tian Y, Min X, Zhai G, Gao Z. Video-based early detection via temporal pyramid networks. In: 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE; 2019:272–277.

  74. Sun K, Li L, Li L, He N, Zhu J. Spatial Attentional Bilinear 3D Convolutional Network for Video-Based Autism Spectrum Disorder Detection. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2020:3387–3391.

  75. Kumdee O, Ritthipravat P. In: 2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT). IEEE. Repetitive motion detection for human behavior understanding from video images. 2015;2015:484–9.

    Google Scholar 

  76. Albinali F, Goodwin MS, Intille S. Detecting stereotypical motor movements in the classroom using accelerometry and pattern recognition algorithms. Pervasive Mob Comput. 2012;8(1):103–14.

    Article  Google Scholar 

  77. Großekathöfer U, Manyakov NV, Mihajlović V, Pandina G, Skalkin A, Ness S, et al. Automated detection of stereotypical motor movements in autism spectrum disorder using recurrence quantification analysis. Front Neuroinform. 2017;11:9.

    Article  Google Scholar 

  78. Kindregan D, Gallagher L, Gormley J. Gait deviations in children with autism spectrum disorders: a review. Autism research and treatment. 2015.

  79. Weiss MJ, Moran MF, Parker ME, Foley JT. Gait analysis of teenagers and young adults diagnosed with autism and severe verbal communication disorders. Front Integr Neurosci. 2013;7:33.

    Article  Google Scholar 

  80. Calhoun M, Longworth M, Chester VL. Gait patterns in children with autism. Clin Biomech. 2011;26(2):200–6.

    Article  Google Scholar 

  81. Hasan CZC, Jailani R, Tahir NM, Yassin IM, Rizman ZI. Automated classification of autism spectrum disorders gait patterns using discriminant analysis based on kinematic and kinetic gait features. Journal of Applied Environmental and Biological Sciences. 2017;7(1):150–6.

    Google Scholar 

  82. Hasan CZC, Jailani R, Tahir NM, Sahak R. Autism spectrum disorders gait identification using ground reaction forces. Telkomnika. 2017;15(2):903.

    Article  Google Scholar 

  83. Hasan C, Jailani R, Tahir N, Desa H. Vertical ground reaction force gait patterns during walking in children with autism spectrum disorders. Int J Eng. 2018;31(5):705–11.

    Google Scholar 

  84. Hasan CZC, Jailani R, Tahir NM. Use of statistical approaches and artificial neural networks to identify gait deviations in children with autism spectrum disorder. Int J Biol Biomed Eng. 2017;11:74–9.

    Google Scholar 

  85. Ebrahimi M, Feghi M, Moradi H, Mirian M, Pouretemad H. Distinguishing tip-toe walking from normal walking using skeleton data gathered by 3D sensors. In: 2015 3rd RSI International Conference on Robotics and Mechatronics (ICROM). IEEE; 2015:450–455.

  86. Ilias S, Tahir NM, Jailani R, Hasan CZC. In: 2016 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE). IEEE. Classification of autism children gait patterns using neural network and support vector machine. 2016;2016:52–6.

    Google Scholar 

  87. Ilias S, Tahir NM, Jailani R, Hasan CZC. In: 2017 European Modelling Symposium (EMS). IEEE. Linear Discriminant Analysis in Classifying Walking Gait of Autistic Children. 2017;2017:67–72.

    Google Scholar 

  88. Ilias S, Tahir NM, Jailani R. In: 2016 IEEE Industrial Electronics and Applications Conference (IEACon). IEEE. Feature extraction of autism gait data using principal component analysis and linear discriminant analysis. 2016;2016:275–9.

    Google Scholar 

  89. Henderson B, Yogarajah P, Gardiner B, McGinnity M, Forster K, Nicholas B. In: 2020 31st Irish Signals and Systems Conference (ISSC). IEEE. Effects of Intra-Subject Variation in Gait Analysis on ASD Classification Performance in Machine Learning Models. 2020;2020:1–6.

    Google Scholar 

  90. Shigeta M, Sawatome A, Ichikawa H, Takemura H. Correlation between Autistic Traits and Gait Characteristics while Two Persons Walk Toward Each Other. Advanced Biomedical Engineering. 2018;7:55–62.

    Article  Google Scholar 

  91. Dufek JS, Eggleston JD, Harry JR, Hickman RA. A comparative evaluation of gait between children with autism and typically developing matched controls. Med Sci. 2017;5(1):1.

    Google Scholar 

  92. Senju A, Johnson MH. The eye contact effect: mechanisms and development. Trends Cogn Sci. 2009;13(3):127–34.

    Article  Google Scholar 

  93. Jiang M, Zhao Q. Learning visual attention to identify people with autism spectrum disorder. In: Proceedings of the IEEE International Conference on Computer Vision; 2017:3267–3276.

  94. Perronnin F, Liu Y, Sánchez J, Poirier H. Large-scale image retrieval with compressed fisher vectors. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE; 2010:3384–3391.

  95. Cho KW, Lin F, Song C, Xu X, Hartley-McAndrew M, Doody KR. In: 2016 IEEE Wireless Health (WH). IEEE. Gaze-Wasserstein: a quantitative screening approach to autism spectrum disorders. 2016;2016:1–8.

    Google Scholar 

  96. Startsev M, Dorr M. Classifying Autism Spectrum Disorder Based on Scanpaths and Saliency. In: 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE; 2019:633–636.

  97. Chen S, Zhao Q. Attention-based autism spectrum disorder screening with privileged modality. In: Proceedings of the IEEE International Conference on Computer Vision; 2019:1181–1190.

  98. Tao Y, Shyu ML. SP-ASDNet: CNN-LSTM based ASD classification model using observer scanpaths. In: 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE; 2019:641–646.

  99. Wan G, Kong X, Sun B, Yu S, Tu Y, Park J, et al. Applying eye tracking to identify autism spectrum disorder in children. J Autism Dev Disord. 2019;49(1):209–15.

    Article  Google Scholar 

  100. Babu PRK, Lahiri U. Classification approach for understanding implications of emotions using eye-gaze. J Ambient Intell Humaniz Comput. 2019;11(7):2701–13.

    Article  Google Scholar 

  101. Nebout A, Wei W, Liu Z, Huang L, LeMeur O. Predicting Saliency Maps for ASD People. In: 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE; 2019:629–632.

  102. Duan H, Zhai G, Min X, Fang Y, Che Z, Yang X, etal. Learning to predict where the children with asd look. In: 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE; 2018:704–708.

  103. Dris AB, Alsalman A, Al-Wabil A, Aldosari M. Intelligent Gaze-Based Screening System for Autism. In: 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS). IEEE; 2019:1–5.

  104. Syeda UH, Zafar Z, Islam ZZ, Tazwar SM, Rasna MJ, Kise K, etal. Visual face scanning and emotion perception analysis between autistic and typically developing children. In: Proceedings of the 2017 acm international joint conference on pervasive and ubiquitous computing and proceedings of the 2017 acm international symposium on wearable computers; 2017:844–853.

  105. Sadria M, Karimi S, Layton AT. Network centrality analysis of eye-gaze data in autism spectrum disorder. Comput Biol Med. 2019;111.

    Article  Google Scholar 

  106. Arru G, Mazumdar P, Battisti F. Exploiting Visual Behaviour for Autism Spectrum Disorder Identification. In: 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE; 2019:637–640.

  107. Liu W, Li M, Yi L. Identifying children with autism spectrum disorder based on their face processing abnormality: A machine learning framework. Autism Res. 2016;9(8):888–98.

    Article  Google Scholar 

  108. Alie D, Mahoor MH, Mattson WI, Anderson DR, Messinger DS. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV). IEEE. Analysis of eye gaze pattern of infants at risk of autism spectrum disorder using markov models. 2011;2011:282–7.

    Google Scholar 

  109. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016:2818–2826.

  110. Gers FA, Schmidhuber JA, Cummins FA. Learning to Forget: Continual Prediction with LSTM. Neural Comput. 2000;12(10):2451–71.

    Article  Google Scholar 

  111. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556. 2014.

  112. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016:770–778.

  113. Ji S, Xu W, Yang M, Yu K. 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell. 2012;35(1):221–31.

    Article  Google Scholar 

  114. Zhuang N, Yusufu T, Ye J, Hua KA. Group activity recognition with differential recurrent convolutional neural networks. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE; 2017:526–531.

  115. Lu Z, Zhou W, Zhang S, Wang C. A new video-based crash detection method: balancing speed and accuracy using a feature fusion deep learning framework. J Adv Transpo. 2020.

  116. Jolliffe IT. Principal Component Analysis. New York: Springer-Verlag; 2002.

    MATH  Google Scholar 

  117. Almeida LB. C1. 2 Multilayer perceptrons. Handbook of Neural Computation C. 1997;1.

  118. Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.

    Article  MATH  Google Scholar 

  119. Swain PH, Hauska H. The decision tree classifier: Design and potential. IEEE Trans Geosci Electron. 1977;15(3):142–7.

    Article  Google Scholar 

  120. Chang CC, Lin CJ. Training v-support vector classifiers: theory and algorithms. Neural Comput. 2001;13(9):2119–47.

    Article  MATH  Google Scholar 

  121. Dalal N, Triggs B. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05). Histograms of oriented gradients for human detection. 2005;2005:886–93.

    Google Scholar 

  122. Oliva A, Torralba A. Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. Int J Comput Vision. 2001;42(3):145–75.

    Article  MATH  Google Scholar 

  123. Pauly L, Sankar D. Detection of drowsiness based on HOG features and SVM classifiers. In: 2015 IEEE International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN). IEEE; 2015:181–186.

  124. Ogaki K, Kitani KM, Sugano Y, Sato Y. Coupling eye-motion and ego-motion features for first-person activity recognition. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE; 2012:1–7.

  125. Wong ET, Yean S, Hu Q, Lee BS, Liu J, Deepu R. Gaze Estimation Using Residual Neural Network. In: 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops); 2019:411–414.

  126. Müller B, Reinhardt J, Strickland MT. Neural Networks. Berlin Heidelberg: Springer; 1995.

    Book  MATH  Google Scholar 

  127. Thabtah F, Peebles D. Early Autism Screening: A Comprehensive Review. Int J Environ Res Public Health. 2019;16(18):3502.

    Article  Google Scholar 

  128. Napolitano DA, Smith T, Zarcone JR, Goodkin K, McAdam DB. Increasing response diversity in children with autism. J Appl Behav Anal. 2010;43(2):265–71.

    Article  Google Scholar 

  129. Whyatt CP, Torres EB. Autism Research: An objective quantitative review of progress and focus between 1994 and 2015. Front Psychol. 2018;9:1526.

    Article  Google Scholar 

  130. Sivalingam R, Cherian A, Fasching J, Walczak N, Bird N, Morellas V, et al. A multi-sensor visual tracking system for behavior monitoring of at-risk children. In: 2012 IEEE International Conference on Robotics and Automation. IEEE; 2012:1345–1350.

  131. McCann J. Youth and Disability: A Challenge to Mr Reasonable. Int J Disabil Dev Educ. 2017;64(6):668–70.

    Article  Google Scholar 

  132. Lord C, Cook EH, Leventhal BL, Amaral DG. Autism Spectrum Disorders. Neuron. 2000;28(2):355–63.

    Article  Google Scholar 

  133. Oosterling IJ, Wensing M, Swinkels SH, Van Der Gaag RJ, Visser JC, Woudenberg T, et al. Advancing early detection of autism spectrum disorder by applying an integrated two-stage screening approach. J Child Psychol Psychiatry. 2010;51(3):250–8.

    Article  Google Scholar 

  134. Thabtah F. Autism spectrum disorder screening: machine learning adaptation and DSM-5 fulfillment. In: Proceedings of the 1st International Conference on Medical and health Informatics. 2017:1–6.

  135. MacDonald JD. Communicating Partners: 30 Years of Building Responsive Relationships with Late Talking Children including Autism, Asperger’s Syndrome (ASD), Down Syndrome, and Typical Devel. Jessica Kingsley Publishers; 2004.

  136. Vishwakarma S, Agrawal A. A survey on activity recognition and behavior understanding in video surveillance. Vis Comput. 2013;29(10):983–1009.

    Article  Google Scholar 

  137. Marszalek M, Laptev I, Schmid C. Actions in context. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009:2929–2936.

  138. Sunny JT, George SM, Kizhakkethottam JJ, Sunny JT, George SM, Kizhakkethottam JJ. Applications and challenges of human activity recognition using sensors in a smart environment. IJIRST Int J Innov Res Sci Technol. 2015;2:50–7.

    Google Scholar 

  139. Wang H, Schmid C. Action recognition with improved trajectories. In: Proceedings of the IEEE international conference on computer vision; 2013:3551–3558.

  140. Kasprowski P, Harężlak K, Stasch M. Guidelines for the eye tracker calibration using points of regard. In: Information Technologies in Biomedicine, Volume 4. Springer; 2014:225–236.

  141. Guillon Q, Hadjikhani N, Baduel S, Rogé B. Visual social attention in autism spectrum disorder: Insights from eye tracking studies. Neuroscience & Biobehavioral Reviews. 2014;42:279–97.

    Article  Google Scholar 

Download references

Funding

This research was supported in part by the ICT Division, Ministry of Posts, Telecommunications and Information Technology of the Government of Bangladesh.

Author information

Authors and Affiliations

  1. Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh

    Sejuti Rahman, Syeda Faiza Ahmed, Omar Shahid & Musabbir Ahmed Arrafi

  2. Osaka University, Osaka, Japan

    M. A. R. Ahad

  3. Department of Electrical & Electronic Engineering, University of Dhaka, Dhaka, Bangladesh

    M. A. R. Ahad

Authors
  1. Sejuti Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Syeda Faiza Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Omar Shahid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Musabbir Ahmed Arrafi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. M. A. R. Ahad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sejuti Rahman.

Ethics declarations

Ethical Approval

This article does not contain any studies with human participants or animals.

Conflict of Interest

All of the authors declare that he/she has no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahman, S., Ahmed, S.F., Shahid, O. et al. Automated Detection Approaches to Autism Spectrum Disorder Based on Human Activity Analysis: A Review. Cogn Comput 14, 1773–1800 (2022). https://doi.org/10.1007/s12559-021-09895-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-021-09895-w

Keywords

Search

Navigation