Skip to main content
Progress in Orthodontics
Download PDF

Applications of artificial intelligence and machine learning in orthodontics: a scoping review

Progress in Orthodontics volume 22, Article number: 18 (2021) Cite this article

Abstract

Introduction

This scoping review aims to provide an overview of the existing evidence on the use of artificial intelligence (AI) and machine learning (ML) in orthodontics, its translation into clinical practice, and what limitations do exist that have precluded their envisioned application.

Methods

A scoping review of the literature was carried out following the PRISMA-ScR guidelines. PubMed was searched until July 2020.

Results

Sixty-two articles fulfilled the inclusion criteria. A total of 43 out of the 62 studies (69.35%) were published this last decade. The majority of these studies were from the USA (11), followed by South Korea (9) and China (7). The number of studies published in non-orthodontic journals (36) was more extensive than in orthodontic journals (26). Artificial Neural Networks (ANNs) were found to be the most commonly utilized AI/ML algorithm (13 studies), followed by Convolutional Neural Networks (CNNs), Support Vector Machine (SVM) (9 studies each), and regression (8 studies). The most commonly studied domains were diagnosis and treatment planning—either broad-based or specific (33), automated anatomic landmark detection and/or analyses (19), assessment of growth and development (4), and evaluation of treatment outcomes (2). The different characteristics and distribution of these studies have been displayed and elucidated upon therein.

Conclusion

This scoping review suggests that there has been an exponential increase in the number of studies involving various orthodontic applications of AI and ML. The most commonly studied domains were diagnosis and treatment planning, automated anatomic landmark detection and/or analyses, and growth and development assessment.

Introduction

The use of information technology (IT) in the dental field has increased significantly over the past 25 years and has helped reduce cost, time, dependence on human expertise, and medical errors. As a subfield of computer science, artificial intelligence (AI) encompasses both hardware and software that can perceive its environment and take action that maximizes its chances of successfully achieving its goals [1,2,3,4]. Although developments in AI had started in 1943 [5], it was only in 1956 that the term was coined by John McCarthy and adopted during a meeting at Dartmouth College [6]. AI allows examination, organization, representation, and cataloging of medical information, and its robust pattern finding and prediction algorithms are helping drive discoveries across all sciences [7]. In 2019, Morgan Stanley estimated that the global market for AI in healthcare could surge from $1.3 billion to $10 billion by 2024, growing at an annual compound rate of 40% [8].

While AI is a broad term and includes various classifications, there are two main categories of AI: symbolic AI and machine learning from an algorithmic perspective. Symbolic AI is a collection of techniques based on structuring the algorithm in a human-readable symbolic manner. This category was the paradigm of AI research until the late 1980s and is widely known as GOFAI—good old-fashioned AI [9]. AI is still used for solving problems in which the possible outcomes are limited, computational power is scarce, or human explainability is essential. However, in healthcare, where problems tend to be complex, not always fully understood, and have many explanatory variables, building a model based on a limited set of rules is extremely difficult, if not impossible [10].

Machine learning (ML)—a term first phrased by Arthur Samuel in 1952, is the current paradigm. The fundamental difference between ML and symbolic AI is that, in ML, the models learn from examples rather than a set of rules established by a human [7]. By utilizing a mixture of statistical and probabilistic tools, machines can learn from previous models and improve their actions when new data is introduced. This could be in the form of predictions, identifying new patterns, or classifying new data. ML can be categorized into three types, depending on the type of learning of the algorithm and the chosen outcome: supervised learning (used for classification or prediction based on a known outcome), unsupervised learning (finding hidden patterns and structures with unknown outcomes), and reinforcement learning (machine develops a modified algorithm based on previous versions that maximizes the intended reward) [11].

Deep learning (DL) is a sub-domain of ML in which the machine itself calculates specific features of a given input. DL’s precursor is an artificial neural network (ANN), which was initially developed in the 1900s. With the exponential increase in computational technology and power, researchers have designed more complicated and “deeper” neural networks to solve more complicated practical problems. The neural network has become known as “deep learning.” [12].

Algorithms used for ML may also be used for data mining. Data mining applies these algorithms to historical data to identify new relationships or patterns and therefore aid practitioners in optimizing decision-making in their daily practice, as well as improving quality of care [13, 14]. Alternatively, if predictions are desired, ML should be utilized. For example, a practitioner can use existing data about a disease to train the machine to calculate predictions about the diagnosis or prognosis of patients that has not yet been seen before. Notably, ML predictive models have been shown to have greater accuracy than statistic-based models [15].

In recent years, scoping reviews have become an increasingly adopted approach and have been published across various social sciences and healthcare fields. Orthodontics has been a slow starter on this terrain [16]. Scoping reviews are of particular use when a body of literature has not yet been comprehensively reviewed or exhibits a large, complex, or heterogeneous nature not amenable to a more thorough systematic review [17].

Current orthodontic literature is replete with studies that have documented various applications of AI and ML, utilizing different types of algorithms mentioned above, albeit in isolation. No study hitherto, has attempted to systematically organize the existing literature to review existing AI and ML applications in orthodontics, classify the types of algorithms applied, and provide a comprehensive mapping of studies conducted in this field. Hence, this scoping review aims to provide an overview of the existing evidence of how far the earlier AI and ML advancements in orthodontics have translated into clinical fruition and the limitations that have precluded their envisioned development. The authors have attempted (1) to chart the evolution of AI in the orthodontic field over the years, (2) to examine the utilization of applications of AI and ML in the field of orthodontics, and (3) to collate the type of artificial intelligence algorithms that have been implemented in orthodontics.

Materials and method

A scoping review of the published literature was performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines. This protocol was not registered previously.

The eligibility criteria of the scoping review are outlined in Table 1.

Table 1 Scoping review eligibility criteria
Full size table

The first author (YMB) conducted an initial literature search in PubMed on 2 July 2020 with the keywords listed in Table 2.

Table 2 Keywords for initial literature search
Full size table

No restrictions were made concerning year or publication status; however, studies with missing English abstracts were excluded. After eliminating duplicate studies, three of the authors (YMB, AYB, NRV) independently screened the titles and abstracts of the retrieved citations to exclude non-eligible articles based on the study’s eligibility criteria and keywords. A copy of the full text was obtained for the articles considered potentially useful after this selection stage. The same authors then read each full-text article to determine whether it met the inclusion criteria. Additional material about the articles included as an appendix was acquired when needed, and the reference lists of acquired articles were also searched for relevant articles. Any disagreement was resolved by discussion between three authors (YMB, AYB, NRV) until a final consensus was achieved.

Data extraction

Data extraction was charted according to “PICO” guidelines and can be found in the supplementary materials. The collected information included the author names, year of publication, country of origin, whether the study was published in an orthodontic specialty journal, study population or sample size, intervention type, comparator, the outcome of the intervention, type of AI algorithm employed, and finally a broad-based orthodontic outcome domain.

Results

The initial search strategy resulted in 289 records, 65 of which were excluded as duplicates, 6 studies that did not appear in the initial search strategy but were identified through references were added subsequently, resulting in 230 records screened for further eligibility. A total of 140 records were excluded as they were irrelevant to the topic of AI/ML applications in orthodontics (n = 108), did not feature an English abstract (n = 17), and finally because the full-text articles of these records could not be retrieved (n = 15). Hence, 90 full-text articles were then evaluated for eligibility, of which case series or case reports (n = 3), personal opinions/descriptive papers/interviews (n = 19), and technique articles/proof of concept (n = 6) were excluded, resulting in 62 articles that fulfilled the inclusion criteria of this study (Fig. 1 illustrates the PRISMA-ScR flowchart for the scoping review). The details of the studies included in the scoping review are listed in Supplementary table 1.

Fig. 1
figure 1

PRISMA flow diagram of the scoping review

Full size image

In regard to the date of publication, 43 out of the 62 studies (69.35%) included in this scoping review were published between 2011 and 2020, 12 studies (19.35%) between 2001 and 2010 and 7 studies (11.3%) were published between 1991 and 2000 (Supplementary table 2). Results reveal that 11 studies originated from the USA, 9 from South Korea, 7 from China, 6 each from Japan and Italy, 4 from Turkey, 3 each from Brazil, India, Germany, and the UK, 2 from Switzerland, Mexico, Colombia, Spain, and 1 study each from Russia, Morocco, Thailand, Iran, Serbia, Singapore, and Australia (Supplementary table 3). Thirty-six out of the 62 studies were published in non-orthodontic journals (58%) whereas 42% (26 studies) were published in orthodontic specialty journals (Supplementary table 4).

Artificial neural networks (ANNs) were found to be utilized as the AI/ML algorithm in 13 studies, convolutional neural networks (CNNs) and support vector machine (SVM) in 9 studies and regression in 8 studies apart from 23 other algorithms utilized in various studies. Results classified as per the type of AI algorithm employed in the study are listed in Table 3.

Table 3 Studies classified by AI/ML algorithm employed
Full size table

The study results also helped classify artificial intelligence applications into 4 core domains and a 5th domain that clubbed together miscellaneous applications. Table 4 enlists results classified as per domains of applications of artificial intelligence in orthodontics.

Table 4 Studies classified as per domains of applications of artificial intelligence in orthodontics
Full size table

Discussion

There has been a steady increase in the number of scoping reviews published in orthodontic literature over the last few years [80, 81]. Through this scoping review, the authors have attempted to organize the existing literature in a systematic manner in order to document existing applications of AI and ML in the field of orthodontics.

The study results reveal that one of the earliest studies to document the utilization of AI in the field of orthodontics was published in 1986 in a non-orthodontic journal; however, this was excluded from this scoping review based on its inclusion criteria [82]. The first study included in this review was found to be published in 1991 [57]; between 1991 and 2000, 2001 and 2010, and 2011 and 2020, there was a progressive increase in publications from 7 to 12 and 43 respectively, clearly indicating an exponential rise in this subject due to technological advancements and the continuous digitization of orthodontics.

These studies were primarily published in non-orthodontic journals (36) compared to orthodontic specialty journals (26). This is perhaps reflective of the far-reaching applications of AI and ML and points out toward possible scope of a further collaboration of various disciplines in the field of AI and ML.

This distribution map of research undertaken in the field of AI in orthodontics shows that the majority of the studies originated from the USA (11), South Korea (7), and Japan and China (7 each). Apart from these 4 countries, studies were also found to originate from 17 other countries, reflecting the more considerable increase in AI and ML’s interest and their subsequent utilization in orthodontics.

This scoping review also tried to examine the types of AI algorithms commonly employed in various studies. The results reveal that artificial neural networks (ANNs) were the widely utilized AI/ML algorithm (10) followed by convolutional neural networks (CNNs), support vector machine (SVM)-8 studies and regression (logistic and linear) in 8 studies apart from 23 other algorithms utilized in various studies.

The research question—“what are AI and ML applications in the field of orthodontics?” threw up five major domains. Each domain was addressed with the PICO framework for literature evaluation and can be enumerated as (1) diagnosis and treatment planning—either broad based or specific, (2) automated anatomic landmark detection and/or analyses, (3) assessment of growth and development, (4) evaluation of treatment outcome and finally a (5) miscellaneous category.

Our study results show that the maximum number of publications focused on automated anatomic landmark detection and/or analyses as the major domain of AI utilization, chiefly from lateral cephalograms, more recently from CBCT images and lastly from frontal cephalograms. This review’s first study to utilize AI for automatic extraction of cephalometric landmarks was published back in 1998 in an orthodontic specialty journal [75]. Several studies since then [27, 32, 35, 37, 64, 65, 69, 70, 74, 77, 78] have affirmed greater accuracy of landmark detection, reduced time, and human effort spent on anatomic landmark detection and/or analyses with AI/ML as compared to traditional methods.

Studies have shown that cephalometric analysis’s angles and lengths, predicted by neural networks, were not statistically significant from those calculated from manually plotted points. Yu et al. [32] proposed a system that exhibited > 90% sensitivity, specificity, and accuracy for vertical and sagittal skeletal diagnosis and concluded that CNN-incorporated system showed potential for skeletal orthodontic diagnosis without the need for intermediary steps requiring complicated diagnostic procedures. The use of CBCT for cephalometric analysis has now become commonplace. Various studies [31, 58,59,60,61, 76] that have employed AI and ML techniques for automatic landmark detection and analysis have shown that the results obtained are as accurate and less time-consuming as compared to those obtained with manual analysis. At least one study [19] included in this review compared frontal cephalometric landmarking ability of humans versus that of artificial neural networks and the results showed that ANNs could achieve accuracy comparable to humans in placing cephalometric points, and in some cases surpasses the accuracy of inexperienced doctors (students, residents, graduate students).

The second domain incorporating AI and ML utilization was broadly labeled as diagnosis and treatment planning, with applications intended either for broad-based or specific clinical situations. Diagnosis remains the cornerstone of successful orthodontic treatment. However, thus far, no tools exist to lead patients and clinicians out of the decision-making uncertainty in which they are trapped, especially when they face a condition that has several possible correct treatment options and orthodontists over the years have attempted to create systems that take the subjective bias out of diagnostic decision-making.

Expert systems [55,56,57, 79] are one of the earliest and most basic implementations of AI and have been popular for diagnosis and treatment planning in the medical and dental fields. They process the input information and provide solutions based on “if-then” rules. “If-then” based expert systems are limited to currently existing data when the system is created, and regular updates are required to ensure that the outcomes is correct and up to date. Rule-based expert systems have now become obsolete due the aforementioned limitations, and the development of newer technologies such as ML.

To “extract or not to extract” has been the question in orthodontics since time immemorial, with substantial variability noted between orthodontists’ decisions. Unsurprisingly, the most significant number of studies [18, 23, 25, 28, 68] under the diagnosis domain was found to investigate the development of several decision support systems that reduce the relative subjectivity or increased complexity of making the extraction decision. Artificial neural networks (ANNs) were used to develop such systems, and they were shown to be successful at predicting the extraction decision with an accuracy of 94% for extraction decision, 84.2% for extraction pattern determination and 92.8% for anchorage pattern determination [23]. This study suggested that ANNs could provide good guidance for orthodontic treatment planning for less experienced orthodontists. Choi et al. [21] expanded the use of ANNs to determine the diagnosis of orthognathic surgery in addition to the extraction decision and their study results showed a 96% success rate for diagnosis of surgery/non-surgery decision and 91% success rate for detailed diagnosis of surgery type and extraction decision. A recent study [18], when comparing all classifiers, concluded that random forest classifier outperforms neural network model for the prediction of the specific extraction treatment.

Four studies included in this scoping review were dedicated toward the assessment of need for orthodontic treatment and/or prediction of treatment outcome [42, 46, 53, 61]. Thanathornwong [53] utilized the Bayesian network (BN) for assessment of the need for orthodontic treatment and concluded that the results obtained by the decision support system were comparable with those suggested by expert orthodontists. Wang et al. [42] explored the function of an eye-tracking method to evaluate orthodontic treatment need and treatment outcome from the lay perspective in an objective way when compared to traditional methods. The authors employed support vector machine techniques and concluded that the eye-tracking device was able to objectively quantify the effect of malocclusion on facial perception and the impact of orthodontic treatment on malocclusion from a lay perspective.

Predictions of treatment outcomes in class II and class III patients have also been reported. Auconi et al. developed a system to predict outcomes in untreated class III patients [62]. Unsupervised learning was used to cluster patients as hypermandibular, hyperdivergent, or balanced based on cephalometric variables. The system was then applied to a treated sample, where it showed that all of the unsuccessful cases belonged to either the hypermandibular or the hyperdivergent cluster. The same author [73] also attempted to identify critical peculiarities of class II and class III malocclusions and demonstrated that class II subjects exhibited few highly connected orthodontic features, while class III patients showed more compact network structure characterized by strong co-occurrence of normal and abnormal clinical functional and radiological features. The study concluded network analysis could allow orthodontists to visually evaluate and anticipate the co-occurrence of auxological anomalies during individual craniofacial growth and possibly localize reactive sites for a therapeutic approach to a malocclusion.

A group of researchers have specifically studied the applications of AI and ML for the detection of TMJ osteoarthritis [22, 47, 66, 67] and have concluded that deep learning neural network was the most accurate method for classification of TMJ-OA that allows disease staging of bony changes in TMJ-OA. The authors expected their efforts to boost future studies into early detection and osteoarthritis patient-specific therapeutic interventions, and thereby improve the health of articular joints.

Three studies in this scoping review focused on the assessment of maxillary constriction and/or maxillary canine impactions [51, 54, 71]. Chen et al. [71] developed a machine learning algorithm utilizing Learning-based multi-source IntegratioN frameworK for segmentation (LINKS) used with CBCT images to quantify volumetric skeletal maxilla discrepancies and suggested palatal expansion could be beneficial for those with unilateral canine impaction, as underdevelopment of maxilla often accompanies canine impaction in early teen years. Another study [51] concluded that among learning machine methods tested to classify data, the best performance was obtained by random forest method, with an overall accuracy of 88.3% in predicting canine eruption. The authors performed measurements on 2D routinely executed radiographic images, found them to be independently related to canine impaction and showed reliable accuracy in predicting maxillary canine eruption. Bayesian network analysis [54] showed bilateral impaction was associated with palatal impactions and longer treatments, pre-treatment alpha-angle was a determinant for the duration of the orthodontic traction and the post-treatment periodontal outcome was not related to pretreatment radiographic variables.

One of the challenges for less experienced orthodontists is the selection of the appropriate treatment modality and appliance. A system was developed to help orthodontists select the appropriate type of headgears [63]. Compared to the selections made by eight expert orthodontists, the system correctly identified the appropriate headgears 95.6% of the time.

Isolated studies included in this scoping review under the domain of diagnosis and treatment planning have also investigated the applications of AI and ML for screening of osteoporosis from panoramic radiographs [34, 41]; assessment of airflow dynamics, prediction of upper airway collapsible sites, and obstructive sleep apnea [40]; prediction of the association between C. difficile infections in hospitalized patients with major surgeries [52]; genetic risk assessment for non-syndromic orofacial cleft patients [48] and for prediction of occurrence of obstructive sleep apnea in patients with Down’s syndrome [72].

The third domain of applicability of AI can be described as the evaluation of orthodontic treatment outcomes and one of the major areas researched includes the effect of orthognathic surgery on facial appearance and age perception [36, 38]. The algorithms used in these studies concluded that most patient’s appearance improved with treatment (66.4%), resulting in a younger appearance of nearly 1 year, especially after a profile altering surgery. Similar improvement was noted on facial attractiveness in 74.7% of patients, especially after lower jaw surge and the authors concluded that AI might be considered to score facial attractiveness and apparent age in orthognathic patients [38].

With regard to the assessment of growth and development and/or evaluation of growth patterns. Spampinato et al. [39] proposed and tested several deep learning approaches to assess skeletal bone age automatically in what was one of the first studies for an automated skeletal bone age assessment, tested on a public dataset and for all age ranges, races, and genders and with a source code available. Results showed an average discrepancy of 0.8 years between manual and automatic assessment and considered to be a state-of-art performance reliability as per the authors. A recent study [20] compared seven artificial intelligence algorithms—k-nearest neighbors, Naïve Bayes, decision tree, artificial neural networks, support vector machines, random forest, and logistic regression algorithms to determine the preferred method of cervical vertebrae maturation and concluded that ANN was the most stable and preferred method of determining the same.

There have been many attempts to aid orthodontists in classifying patient growth patterns [83,84,85]. One of the first methods of using ANN in evaluating growth occurred in 1998 where the growth of 43 untreated children was classified based on size and shape changes [30]. Nino-Sandoval et al. utilized a support vector machine to classify skeletal patterns through craniomaxillary variables but achieved only 74.51 % accuracy in the correct distinction of class II skeletal pattern from class III pattern and vice-versa [24, 44].

Applications of AI and ML that could not be described under the above four major domains were grouped under the miscellaneous category in this review and these include automated tooth segmentation either from CBCT images or dental models [33, 43], detection of activation pattern of tongue musculature [50] and evaluation of effects of a different curing unit and light-tips on temperature increase during orthodontic bonding [26].

Limitations

Some AI applications may have been missed out due to the inclusion criteria, utilized search terms, if they were published in a language other than English and/or due to the non-inclusion in PubMed.

Due to its nature, scoping reviews are not expected to utilize a risk of bias tool to assess methodological strengths of included studies. The overall idea is to explore in a superficial way what is currently known in a specific area.

Conclusion

  • This scoping review showcases that there has been an exponential increase in the number of orthodontic studies involving various applications of AI and ML over the past three decades.

  • The majority of these studies originated from the USA, followed by South Korea and China.

  • The number of studies published in non-orthodontic journals (36) was found to be more extensive than those published in orthodontic specialty journals (26).

  • Artificial neural networks (ANNs) were found to be the most commonly utilized AI/ML algorithm, followed by convolutional neural networks (CNNs) and support vector machine (SVM).

  • The most commonly utilized AI domains were for diagnosis and treatment planning (33 studies), automated anatomic landmark detection and/or analyses (19), assessment of growth and development (4), and evaluation of treatment outcome (2).

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

IT:

Information technology

AI:

Artificial intelligence

ML:

Machine learning

DL:

Deep learning

References

  1. Asiri SN, Tadlock LP, Schneiderman E, Buschang PH. Applications of artificial intelligence and machine learning in orthodontics. APOS Trends Orthod. 2020;10(1):17–24. https://doi.org/10.25259/APOS_117_2019.

    Article  Google Scholar 

  2. Haugeland J. Artificial intelligence: the very idea. Cambridge: Massachusetts Insitute of Technology; 1985.

    Google Scholar 

  3. Morris CG. Academic Press Dictionary of Science Technology. San Diego: Academic Press; 1996.

    Google Scholar 

  4. Luger GF. Artificial intelligence: structures and strategies for complex problem solving. Harlow: Pearson Education; 2005.

    Google Scholar 

  5. McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys. 1943;5(4):115–33. https://doi.org/10.1007/BF02478259.

    Article  Google Scholar 

  6. Negnevitsky M. Artificial intelligence: a guide to intelligent systems. Pearson Education: Canada; 2005.

    Google Scholar 

  7. Faber J, Faber C, Faber P. Artificial intelligence in orthodontics. APOS Trends Orthod. 2019;9(4):201–5. https://doi.org/10.25259/APOS_123_2019.

    Article  Google Scholar 

  8. Could Artificial Intelligence Transform Healthcare? Morgan Stanley. Available from: https://www.morganstanley.com/ideas/medtech-artificial-intelligence. Accessed 12 Dec 2020.

  9. Michael W, Haugeland J. Artificial intelligence: the very idea. Technol Culture. 1987;28(4):905–22.

    Google Scholar 

  10. Schwartz WB, Patil RS, Szolovits P. Artificial intelligence in medicine. Where do we stand? N Engl J Med. 1987;316(11):685–8. https://doi.org/10.1056/NEJM198703123161109.

    Article  PubMed  Google Scholar 

  11. Bishop CM. Pattern recognition and machine learning. New York: Springer; 2006.

    Google Scholar 

  12. Ko C, Tanikawa C, Wu T, Pastewait Matthew, Jackson C B, Kwon JJ, Lee Y, Lian C, Wang Li, Shen D Machine learning in orthodontics: application review. Embracing novel technologies in dentistry and orthodontics- Forty-sixth Annual Moyers Symposium and the Forty-fourth Annual International Conference on Craniofacial Research March 1-3, 2019, Ann Arbor, Michigan.

  13. Sumathi S, Sivanandam S. Introduction to data mining principles. Introduction to Data Mining and its Applications. Studies in Computational Intelligence. Berlin, Heidelberg: Springer; 2006. p. 1–20.

    Book  Google Scholar 

  14. Mitchell TM. Machine learning and data mining. Commun ACM. 1999;42:1–13.

    Article  Google Scholar 

  15. Breiman L. Statistical modeling: the two cultures (with comments and a rejoinder by the author). Stat Sci. 2001;16:199–231.

    Article  Google Scholar 

  16. Vaid N. Scoping studies: should there be more in orthodontic literature? APOS Trends Orthod. 2019;9(3):124–5. https://doi.org/10.25259/APOS_118_2019.

    Article  Google Scholar 

  17. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. https://doi.org/10.1080/1364557032000119616.

    Article  Google Scholar 

  18. Suhail Y, Upadhyay M, Chhibber A, Kshitiz. Machine learning for the diagnosis of orthodontic extractions: a computational analysis using ensemble learning. Bioengineering. 2020;7:55.

    Article  Google Scholar 

  19. Muraev AA, Tsai P, Kibardin I, Oborotistov N, Shirayeva T, Ivanov S, et al. Frontal cephalometric landmarking: humans vs artificial neural networks. Int J Comput Dent. 2020;23(2):139–48.

    PubMed  Google Scholar 

  20. Kök H, Acilar AM, İzgi MS. Usage and comparison of artificial intelligence algorithms for determination of growth and development by cervical vertebrae stages in orthodontics. Prog Orthod. 2019;20(1):41. https://doi.org/10.1186/s40510-019-0295-8.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Choi HI, Jung SK, Baek SH, Lim WH, Ahn SJ, Yang IH, et al. Artificial intelligent model with neural network machine learning for the diagnosis of orthognathic surgery. J Craniofac Surg. 2019;30(7):1986–9. https://doi.org/10.1097/SCS.0000000000005650 Erratum in: J Craniofac Surg. 2020 Jun;31(4):1156.

    Article  PubMed  Google Scholar 

  22. Shoukri B, Prieto JC, Ruellas A, Yatabe M, Sugai J, Styner M, et al. Minimally invasive approach for diagnosing TMJ osteoarthritis. J Dent Res. 2019;98(10):1103–11. https://doi.org/10.1177/0022034519865187 Epub 2019 Jul 24.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Li P, Kong D, Tang T, Su D, Yang P, Wang H, et al. Orthodontic treatment planning based on artificial neural networks. Sci Rep. 2019;9(1):2037. https://doi.org/10.1038/s41598-018-38439-w.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Niño-Sandoval TC, Guevara Pérez SV, González FA, Jaque RA, Infante-Contreras C. Use of automated learning techniques for predicting mandibular morphology in skeletal class I, II and III. Forensic Sci Int. 2017;281:187.e1–7. https://doi.org/10.1016/j.forsciint.2017.10.004 Epub 2017 Oct 12.

    Article  Google Scholar 

  25. Jung SK, Kim TW. New approach for the diagnosis of extractions with neural network machine learning. Am J Orthod Dentofacial Orthop. 2016;149(1):127–33. https://doi.org/10.1016/j.ajodo.2015.07.030.

    Article  PubMed  Google Scholar 

  26. Aksakalli S, Demir A, Selek M, Tasdemir S. Temperature increase during orthodontic bonding with different curing units using an infrared camera. Acta Odontol Scand. 2014;72(1):36–41. https://doi.org/10.3109/00016357.2013.794954 Epub 2013 May 3. PMID: 23638766.

    Article  PubMed  Google Scholar 

  27. Mario MC, Abe JM, Ortega NR, Del Santo M Jr. Paraconsistent artificial neural network as auxiliary in cephalometric diagnosis. Artif Organs. 2010;34(7):E215–21. https://doi.org/10.1111/j.1525-1594.2010.00994.x Epub 2010 Jun 2.

    Article  PubMed  Google Scholar 

  28. Xie X, Wang L, Wang A. Artificial neural network modeling for deciding if extractions are necessary prior to orthodontic treatment. Angle Orthod. 2010;80(2):262–6. https://doi.org/10.2319/111608-588.1.

    Article  PubMed  Google Scholar 

  29. Akdenur B, Okkesum S, Kara S, Günes S. Correlation- and covariance-supported normalization method for estimating orthodontic trainer treatment for clenching activity. Proc Inst Mech Eng H. 2009;223(8):991–1001. https://doi.org/10.1243/09544119JEIM619.

    Article  PubMed  Google Scholar 

  30. Lux CJ, Stellzig A, Volz D, Jäger W, Richardson A, Komposch G. A neural network approach to the analysis and classification of human craniofacial growth. Growth Dev Aging. 1998;62(3):95–106.

    PubMed  Google Scholar 

  31. Ma Q, Kobayashi E, Fan B, Nakagawa K, Sakuma I, Masamune K, et al. Automatic 3D landmarking model using patch-based deep neural networks for CT image of oral and maxillofacial surgery. Int J Med Robot. 2020;16(3):e2093. https://doi.org/10.1002/rcs.2093 Epub 2020 Mar 20.

    Article  PubMed  Google Scholar 

  32. Yu HJ, Cho SR, Kim MJ, Kim WH, Kim JW, Choi J. Automated skeletal classification with lateral cephalometry based on artificial intelligence. J Dent Res. 2020;99(3):249–56. https://doi.org/10.1177/0022034520901715 Epub 2020 Jan 24.

    Article  PubMed  Google Scholar 

  33. Chung M, Lee M, Hong J, Park S, Lee J, Lee J, et al. Pose-aware instance segmentation framework from cone beam CT images for tooth segmentation. Comput Biol Med. 2020;120:103720. https://doi.org/10.1016/j.compbiomed.2020.103720 Epub 2020 Mar 28.

    Article  PubMed  Google Scholar 

  34. Lee KS, Jung SK, Ryu JJ, Shin SW, Choi J. Evaluation of transfer learning with deep convolutional neural networks for screening osteoporosis in dental panoramic radiographs. J Clin Med. 2020;9(2):392. https://doi.org/10.3390/jcm9020392 PMID: 32024114.

    Article  PubMed Central  Google Scholar 

  35. Kunz F, Stellzig-Eisenhauer A, Zeman F, Boldt J. Artificial intelligence in orthodontics: evaluation of a fully automated cephalometric analysis using a customized convolutional neural network. J Orofac Orthop. 2020;81(1):52-68. English. doi: https://doi.org/10.1007/s00056-019-00203-8. Epub 2019 Dec 18.

  36. Patcas R, Timofte R, Volokitin A, Agustsson E, Eliades T, Eichenberger M, et al. Facial attractiveness of cleft patients: a direct comparison between artificial-intelligence-based scoring and conventional rater groups. Eur J Orthod. 2019;41(4):428–33. https://doi.org/10.1093/ejo/cjz007.

    Article  PubMed  Google Scholar 

  37. Nishimoto S, Sotsuka Y, Kawai K, Ishise H, Kakibuchi M. Personal computer-based cephalometric landmark detection with deep learning, using cephalograms on the internet. J Craniofac Surg. 2019;30(1):91–5. https://doi.org/10.1097/SCS.0000000000004901.

    Article  PubMed  Google Scholar 

  38. Patcas R, Bernini DAJ, Volokitin A, Agustsson E, Rothe R, Timofte R. Applying artificial intelligence to assess the impact of orthognathic treatment on facial attractiveness and estimated age. Int J Oral Maxillofac Surg. 2019;48(1):77–83. https://doi.org/10.1016/j.ijom.2018.07.010 Epub 2018 Aug 4.

    Article  PubMed  Google Scholar 

  39. Spampinato C, Palazzo S, Giordano D, Aldinucci M, Leonardi R. Deep learning for automated skeletal bone age assessment in X-ray images. Med Image Anal. 2017;36:41–51. https://doi.org/10.1016/j.media.2016.10.010 Epub 2016 Oct 29.

    Article  PubMed  Google Scholar 

  40. Yeom SH, Na JS, Jung HD, Cho HJ, Choi YJ, Lee JS. Computational analysis of airflow dynamics for predicting collapsible sites in the upper airways: machine learning approach. J Appl Physiol (1985). 2019;127(4):959–73. https://doi.org/10.1152/japplphysiol.01033.2018 Epub 2019 Jul 18.

    Article  Google Scholar 

  41. Hwang JJ, Lee JH, Han SS, Kim YH, Jeong HG, Choi YJ, et al. Strut analysis for osteoporosis detection model using dental panoramic radiography. Dentomaxillofac Radiol. 2017;46(7):20170006. https://doi.org/10.1259/dmfr.20170006 Epub 2017 Jul 14.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Wang X, Cai B, Cao Y, Zhou C, Yang L, Liu R, et al. Objective method for evaluating orthodontic treatment from the lay perspective: an eye-tracking study. Am J Orthod Dentofacial Orthop. 2016;150(4):601–10. https://doi.org/10.1016/j.ajodo.2016.03.028.

    Article  PubMed  Google Scholar 

  43. Mortaheb P, Rezaeian M. Metal artifact reduction and segmentation of dental computerized tomography images using least square support vector machine and mean shift algorithm. J Med Signals Sens. 2016;6(1):1–11. https://doi.org/10.4103/2228-7477.175867.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Niño-Sandoval TC, Guevara Perez SV, González FA, Jaque RA, Infante-Contreras C. An automatic method for skeletal patterns classification using craniomaxillary variables on a Colombian population. Forensic Sci Int. 2016261:159.e1-159.e6. doi: https://doi.org/10.1016/j.forsciint.2015.12.025. Epub 2015 Dec 25.

  45. Yu X, Liu B, Pei Y, Xu T. Evaluation of facial attractiveness for patients with malocclusion: a machine-learning technique employing Procrustes. Angle Orthod. 2014;84(3):410–6. https://doi.org/10.2319/071513-516.1 Epub 2013 Oct 3.

    Article  PubMed  Google Scholar 

  46. Kim BM, Kang BY, Kim HG, Baek SH. Prognosis prediction for class III malocclusion treatment by feature wrapping method. Angle Orthod. 2009;79(4):683–91. https://doi.org/10.2319/071508-371.1.

    Article  PubMed  Google Scholar 

  47. Bianchi J, de Oliveira Ruellas AC, Gonçalves JR, Paniagua B, Prieto JC, Styner M, et al. Osteoarthritis of the temporomandibular joint can be diagnosed earlier using biomarkers and machine learning. Sci Rep. 2020;10(1):8012. https://doi.org/10.1038/s41598-020-64942-0.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Zhang SJ, Meng P, Zhang J, Jia P, Lin J, Wang X, et al. Machine learning models for genetic risk assessment of infants with non-syndromic orofacial cleft. Genomics Proteomics Bioinformatics. 2018;16(5):354–64. https://doi.org/10.1016/j.gpb.2018.07.005 Epub 2018 Dec 19.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Sorihashi Y, Stephens CD, Takada K. An inference modeling of human visual judgment of sagittal jaw-base relationships based on cephalometry: part II. Am J Orthod Dentofacial Orthop. 2000;117(3):303–11. https://doi.org/10.1016/s0889-5406(00)70235-6.

    Article  PubMed  Google Scholar 

  50. Tolpadi AA, Stone ML, Carass A, Prince JL, Gomez AD. Inverse biomechanical modeling of the tongue via machine learning and synthetic training data. Proc SPIE Int Soc Opt Eng. 2018;10576:1057606. https://doi.org/10.1117/12.2296927 Epub 2018 Mar 12.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Laurenziello M, Montaruli G, Gallo C, Tepedino M, Guida L, Perillo L, et al. Determinants of maxillary canine impaction: retrospective clinical and radiographic study. J Clin Exp Dent. 2017;9(11):e1304–9. https://doi.org/10.4317/jced.54095.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Allareddy V, Wang T, Rampa S, Caplin J, Nalliah R, Badheka A, et al. Prevalence and predictors of C. difficile infections in hospitalized patients with major surgical procedures in the USA: analysis using traditional and machine learning methods. Am J Surg. 2019;218(3):661–2. https://doi.org/10.1016/j.amjsurg.2018.11.014 Epub 2018 Dec 1.

    Article  PubMed  Google Scholar 

  53. Thanathornwong B. Bayesian-based decision support system for assessing the needs for orthodontic treatment. Healthc Inform Res. 2018;24(1):22–8. https://doi.org/10.4258/hir.2018.24.1.22 Epub 2018 Jan 31.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Nieri M, Crescini A, Rotundo R, Baccetti T, Cortellini P, Pini Prato GP. Factors affecting the clinical approach to impacted maxillary canines: a Bayesian network analysis. Am J Orthod Dentofacial Orthop. 2010;137(6):755–62. https://doi.org/10.1016/j.ajodo.2008.08.028.

    Article  PubMed  Google Scholar 

  55. Stephens C, Mackin N. The validation of an orthodontic expert system rule-base for fixed appliance treatment planning. Eur J Orthod. 1998;20(5):569–78. https://doi.org/10.1093/ejo/20.5.569.

    Article  PubMed  Google Scholar 

  56. Stephens CD, Mackin N, Sims-Williams JH. The development and validation of an orthodontic expert system. Br J Orthod. 1996;23(1):1–9. 8652492. https://doi.org/10.1179/bjo.23.1.1.

    Article  PubMed  Google Scholar 

  57. Brown ID, Adams SR, Stephens CD, Erritt SJ, Sims-Williams JH. The initial use of a computer-controlled expert system in the treatment planning of class II division 1 malocclusion. Br J Orthod. 1991;18(1):1–7. https://doi.org/10.1179/bjo.18.1.1.

    Article  PubMed  Google Scholar 

  58. Montúfar J, Romero M, Scougall-Vilchis RJ. Hybrid approach for automatic cephalometric landmark annotation on cone-beam computed tomography volumes. Am J Orthod Dentofacial Orthop. 2018;154(1):140–50. https://doi.org/10.1016/j.ajodo.2017.08.028.

    Article  PubMed  Google Scholar 

  59. Montúfar J, Romero M, Scougall-Vilchis RJ. Automatic 3-dimensional cephalometric landmarking based on active shape models in related projections. Am J Orthod Dentofacial Orthop. 2018;153(3):449–58. https://doi.org/10.1016/j.ajodo.2017.06.028.

    Article  PubMed  Google Scholar 

  60. Gupta A, Kharbanda OP, Sardana V, Balachandran R, Sardana HK. Accuracy of 3D cephalometric measurements based on an automatic knowledge-based landmark detection algorithm. Int J Comput Assist Radiol Surg. 2016;11(7):1297–309. https://doi.org/10.1007/s11548-015-1334-7 Epub 2015 Dec 24.

    Article  PubMed  Google Scholar 

  61. Gupta A, Kharbanda OP, Sardana V, Balachandran R, Sardana HK. A knowledge-based algorithm for automatic detection of cephalometric landmarks on CBCT images. Int J Comput Assist Radiol Surg. 2015;10(11):1737–52. https://doi.org/10.1007/s11548-015-1173-6 Epub 2015 Apr 7.

    Article  PubMed  Google Scholar 

  62. Auconi P, Scazzocchio M, Cozza P, McNamara JA Jr, Franchi L. Prediction of class III treatment outcomes through orthodontic data mining. Eur J Orthod. 2015;37(3):257–67. https://doi.org/10.1093/ejo/cju038 Epub 2014 Sep 4.

    Article  PubMed  Google Scholar 

  63. Akçam MO, Takada K. Fuzzy modelling for selecting headgear types. Eur J Orthod. 2002;24(1):99–106. https://doi.org/10.1093/ejo/24.1.99.

    Article  PubMed  Google Scholar 

  64. Vucinić P, Trpovski Z, Sćepan I. Automatic landmarking of cephalograms using active appearance models. Eur J Orthod. 2010;32(3):233–41. https://doi.org/10.1093/ejo/cjp099 Epub 2010 Mar 4.

    Article  PubMed  Google Scholar 

  65. Rueda S, Alcañiz M. An approach for the automatic cephalometric landmark detection using mathematical morphology and active appearance models. Med Image Comput Comput Assist Interv. 2006;9(Pt 1):159–66. https://doi.org/10.1007/11866565_20.

    Article  PubMed  Google Scholar 

  66. Ribera NT, de Dumast P, Yatabe M, Ruellas A, Ioshida M, Paniagua B, et al. Shape variation analyzer: a classifier for temporomandibular joint damaged by osteoarthritis. Proc SPIE Int Soc Opt Eng. 2019;10950:1095021. https://doi.org/10.1117/12.2506018 Epub 2019 Mar 13.

    Article  PubMed  PubMed Central  Google Scholar 

  67. de Dumast P, Mirabel C, Cevidanes L, Ruellas A, Yatabe M, Ioshida M, et al. A web-based system for neural network based classification in temporomandibular joint osteoarthritis. Comput Med Imaging Graph. 2018;67:45–54. https://doi.org/10.1016/j.compmedimag.2018.04.009 Epub 2018 May 1.

    Article  PubMed  PubMed Central  Google Scholar 

  68. Takada K, Yagi M, Horiguchi E. Computational formulation of orthodontic tooth-extraction decisions. Part I: to extract or not to extract. Angle Orthod. 2009;79(5):885–91. https://doi.org/10.2319/081908-436.1.

    Article  PubMed  Google Scholar 

  69. Grau V, Alcañiz M, Juan MC, Monserrat C, Knoll C. Automatic localization of cephalometric Landmarks. J Biomed Inform. 2001;34(3):146–56. https://doi.org/10.1006/jbin.2001.1014.

    Article  PubMed  Google Scholar 

  70. Kim H, Shim E, Park J, Kim YJ, Lee U, Kim Y. Web-based fully automated cephalometric analysis by deep learning. Comput Methods Programs Biomed. 2020;194:105513. https://doi.org/10.1016/j.cmpb.2020.105513 Epub 2020 May 6.

    Article  PubMed  Google Scholar 

  71. Chen S, Wang L, Li G, Wu TH, Diachina S, Tejera B, et al. Machine learning in orthodontics: introducing a 3D auto-segmentation and auto-landmark finder of CBCT images to assess maxillary constriction in unilateral impacted canine patients. Angle Orthod. 2020;90(1):77–84. https://doi.org/10.2319/012919-59.1 Epub 2019 Aug 12.

    Article  PubMed  Google Scholar 

  72. Skotko BG, Macklin EA, Muselli M, Voelz L, McDonough ME, Davidson E, et al. A predictive model for obstructive sleep apnea and Down syndrome. Am J Med Genet A. 2017;173(4):889–96. https://doi.org/10.1002/ajmg.a.38137 Epub 2017 Jan 26.

    Article  PubMed  Google Scholar 

  73. Auconi P, Caldarelli G, Scala A, Ierardo G, Polimeni A. A network approach to orthodontic diagnosis. Orthod Craniofac Res. 2011;14(4):189–97. https://doi.org/10.1111/j.1601-6343.2011.01523.x.

    Article  PubMed  Google Scholar 

  74. Tanikawa C, Yagi M, Takada K. Automated cephalometry: system performance reliability using landmark-dependent criteria. Angle Orthod. 2009;79(6):1037–46. https://doi.org/10.2319/092908-508R.1.

    Article  PubMed  Google Scholar 

  75. Rudolph DJ, Sinclair PM, Coggins JM. Automatic computerized radiographic identification of cephalometric landmarks. Am J Orthod Dentofacial Orthop. 1998;113(2):173–9. https://doi.org/10.1016/s0889-5406(98)70289-6.

    Article  PubMed  Google Scholar 

  76. Ed-Dhahraouy M, Riri H, Ezzahmouly M, Bourzgui F, El Moutaoukkil A. A new methodology for automatic detection of reference points in 3D cephalometry: a pilot study. Int Orthod. 2018;16(2):328–37. https://doi.org/10.1016/j.ortho.2018.03.013 Epub 2018 Apr 6.

    Article  PubMed  Google Scholar 

  77. Neelapu BC, Kharbanda OP, Sardana V, Gupta A, Vasamsetti S, Balachandran R, et al. Automatic localization of three-dimensional cephalometric landmarks on CBCT images by extracting symmetry features of the skull. Dentomaxillofac Radiol. 2018;47(2):20170054. https://doi.org/10.1259/dmfr.20170054 Epub 2018 Jan 3.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Tanikawa C, Yamamoto T, Yagi M, Takada K. Automatic recognition of anatomic features on cephalograms of preadolescent children. Angle Orthod. 2010;80(5):812–20. https://doi.org/10.2319/092909-474.1.

    Article  PubMed  Google Scholar 

  79. Hammond RM, Freer TJ. Application of a case-based expert system to orthodontic diagnosis and treatment planning. Aust Orthod J. 1997;14(4):229–34.

    PubMed  Google Scholar 

  80. Gandedkar NH, Vaid NR, Darendeliler MA, Premjani P, Ferguson DJ. The last decade in orthodontics: a scoping review of the hits, misses and the near misses! Semin in Orthod. Vol. 2019;25(4):339–55.

    Google Scholar 

  81. Vaid NR, Hansa I, Bichu Y. Smartphone applications used in orthodontics: a scoping review of scholarly literature. J World Fed Orthod. 2020;9(3S):S67–73. https://doi.org/10.1016/j.ejwf.2020.08.007 Epub 2020 Sep 30.

    Article  PubMed  Google Scholar 

  82. Lévy-Mandel AD, Venetsanopoulos AN, Tsotsos JK. Knowledge-based landmarking of cephalograms. Comput Biomed Res. 1986;19(3):282–309. https://doi.org/10.1016/0010-4809(86)90023-6.

    Article  PubMed  Google Scholar 

  83. Brodie AG. Behavior of normal and abnormal facial growth patterns. Am J Orthod Dent Orthop. 1941;27:633–47.

    Google Scholar 

  84. Broadbent BH. The face of the normal child. Angle Orthod. 1937;7:183–208.

    Google Scholar 

  85. Ricketts RM. A principle of arcial growth of the mandible. Angle Orthod. 1972;42(4):368–86. https://doi.org/10.1043/0003-3219(1972)042<0368:APOAGO>2.0.CO;2.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

N/A.

Funding

No funding was obtained

Author information

Authors and Affiliations

  1. Abu Dhabi, United Arab Emirates

    Yashodhan M. Bichu & Aditi Y. Bichu

  2. Durban, South Africa

    Ismaeel Hansa

  3. Dubai, United Arab Emirates

    Pratik Premjani

  4. Department of Orthodontics, University of Alberta, Edmonton, Alberta, Canada

    Carlos Flores-Mir

  5. Department of Orthodontics, European University College, Dubai, United Arab Emirates

    Nikhilesh R. Vaid

Authors
  1. Yashodhan M. Bichu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ismaeel Hansa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aditi Y. Bichu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pratik Premjani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carlos Flores-Mir
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nikhilesh R. Vaid
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YMB and AYB carried out the data collection and wrote the manuscript. IH aided in data collection and helped draft the manuscript. CFM and PP helped with design and manuscript editing. NRV conceived the study and participated in its design, coordination, and manuscript. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Ismaeel Hansa.

Ethics declarations

Ethics approval and consent to participate

N/A.

Consent for publication

N/A.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary table 1.

Master chart of studies included in the scoping review.

Additional file 2: Supplementary table 2.

Distribution of studies per decade.

Additional file 3: Supplementary table 3.

Country of origin of the study.

Additional file 4: Supplementary table 4.

Studies published in orthodontic specialty journals or non-orthodontic journals.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bichu, Y.M., Hansa, I., Bichu, A.Y. et al. Applications of artificial intelligence and machine learning in orthodontics: a scoping review. Prog Orthod. 22, 18 (2021). https://doi.org/10.1186/s40510-021-00361-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40510-021-00361-9

Keywords