figure b

Introduction

Diabetic foot disease is one of the most serious complications of diabetes mellitus. It represents a cluster of syndromes in which neuropathy, ischaemia and infection can lead to destruction of the foot and can result in severe morbidity and limb amputation. Consequently, diabetic foot disease poses a huge burden on people with diabetes, their families, healthcare systems and society in general [1]. Strategies that include elements of screening, prevention, patient and staff education and interdisciplinary treatment can reduce the load of diabetic foot disease at an individual and public level [2]. However, implementation of guidelines and the establishment of multidisciplinary clinics for holistic management of diabetic foot disorders varies and remains suboptimal [1].

Absence of symptoms does not exclude foot disease (neuropathy, peripheral artery disease, pre-ulcerative signs or even an ulcer) in people with diabetes. Self-assessment of diabetes-related foot problems is unreliable, and self-perceived foot health should be assessed together with foot examination findings [3]. The International Working Group on the Diabetic Foot (IWGDF) recommends an annual examination of people with diabetes at very low risk of foot ulceration (IWGDF risk 0) to screen for developing risk factors [2]. The recommendation of at least a 12-month interval between screenings is in line with the view of other scientific societies [4] and is largely based on the general expert consensus and on the results of the study of Litzelman et al [5], which did not directly address the question of different lengths of the screening interval.

A large proportion of people with diabetes do not receive proper screening due to insufficiencies in healthcare resources, even in developed countries. It is estimated that only 20% of people with diabetes are being screened with a monofilament according to the Echantillon national representative des diabétiques (ENTRED) study in France [6]. A multicentre, epidemiological, cross-sectional study from Alonso-Fernandéz et al [7], which examined clinical records of 443 people with type 2 diabetes in primary care in Spain, showed poor diabetic foot screening compliance (performed in 37%) and infrequent ulcer risk stratification (performed in 12.4%). According to the Scottish Diabetes Survey from 2019, 56.7% of people with type 1 diabetes and 64.8% with type 2 diabetes had their foot scores recorded within 15 months [8]. Alternative forms of screening have been studied, with the aim of lowering the substantial financial cost and capacity burden for healthcare from extensive screening according to guidelines [9, 10].

The probability of foot complications among people with diabetes varies substantially from individual to individual and within different populations [11]. This variation is not addressed efficiently by current recommendations, which propose fixed and arbitrary screening frequency [12]. A clinical prediction model could be used to tailor the frequency of screening for each person based on individual probability of complications at the time of screening. The main aim of this study was to construct a prediction model based on individual risk estimation with the use of penalised logistic regression. Additionally, we aimed at evaluating an optimised process for screening of diabetic foot, based on the developed model.

Methods

Study population and data source

The present study was performed on data collected from electronic health records (EHRs), which were generated during routine work-flow in a single tertiary diabetes clinic. Extensive clinical data from consecutive routine diabetes foot screenings carried out between 2015 and 2020 were prospectively collected in the local database. A detailed structured form was completed at each visit, including history of symptoms, foot inspection, monofilament testing and palpation of peripheral pulses. Examination of light touch sensation with a standardised 10 g Semmes–Weinstein monofilament test (SWM) was used to detect loss of protective sensation (LOPS) [13], which is a significant risk factor for diabetic foot ulceration. LOPS was defined as more than two insensate points to SWM testing (using a ten-point protocol) on either foot. Loss of peripheral pulses (LPP) was defined as at least one impalpable pedal pulse on either foot. The presence of foot symptoms (pain, tingling, numbness or restless legs) was recorded at each visit. Additionally, data on foot deformities (hallux valgus, muscle atrophy, nail deformity or claw toe), current or previous foot ulcer, and Charcot foot history were collected. The records of those with any previous amputation were excluded from analysis because of the inability to apply the ten-point SWM.

Data preparation

Data were prepared and analysed using R language for statistical computing (version 4.0.4) [14] according to study protocol. The study was approved by the National Medical Ethics Committee of Slovenia (reference no. 0120-36/2020/4). The primary foot examinations dataset was extended with results of routine clinical and laboratory measurements (if they were available in our electronic database) within a 1 year period before foot examination. Only screening records with complete information about duration and type of diabetes were included in the sample (Table 1, Fig. 1).

Table 1 Variables used for model creation
Full size table
Fig. 1
figure 1

Flow chart of data sample generation

Full size image

Model generation

One of the main goals of our study was to create a prediction model that can be used in practice to estimate probability of LOPS and LPP at the time of screening. Two of the key characteristics of predictive models are accuracy and generalisability. Accuracy refers to the ability of the model to correctly predict an outcome, whereas generalisability refers to the ability of the model to confidently predict outcomes with unseen data [15].

The concept of over-fitting refers to the tendency for a model to perform worse on a new population. When a model is over-fitted, it is likely to be accurate in the dataset in which it was developed but is unlikely to generalise well to new data [15]. In comparison with traditional regression methods, penalised regression models improve prediction in new data by reducing the size of the estimated coefficients. The penalty parameters λ and α control how much emphasis is given to the penalty. The higher the λ value, the more coefficients in the regression will be pushed towards zero. The model becomes less sensitive to the characteristics of training data with such modification, resulting in smaller changes in predictions when estimating the same model in new data [15].

The penalised logistic regression prediction model [16] was developed to predict LOPS and LPP from the dataset with basic demographics and diabetes, foot investigation and routine visit data (Table 1). Additionally, LOPS and LPP were incorporated into the model for prediction of LOPS and LPP, respectively. R package glmnet (version 4.0-2) [17] was used for model generation, and predictors were standardised using the default standardisation method provided by glmnet function. Optimal penalty parameters were selected by maximising the area under the receiver characteristic operating curve and leave-one-out cross validation (LOOCV). Discrimination parameters and coefficients for logistic calibration analysis were obtained for the assessment of accuracy of probability estimates [18]. We used ‘grouped’ LOOCV (i.e. all observations from the same individual were left out from the model- building phase). The event probability for the observations for the left-out individual were then estimated using the parameter estimates obtained from the model excluding all observations from this individual.

The probabilities for LPP and LOPS were calculated for all individuals classified as IWGDF risk 0 at their foot screening visit, using the optimal classifier with leave-one-out method. Progression risk for LOPS or LPP within 5 years after foot examination was estimated. The calculation of each individual’s probability was based upon prediction models generated by the leave-one-out method at the time of examination. This was then recalculated for the subsequent 5 years for each individual with modification of predictors age and duration of diabetes; all other predictors were left the same as at the time of screening.

Evaluation of a proof-of-concept organisational model

Clinical information gathered at each screening visit could potentially be used for the calculation of personalised risk, thus modifying the subsequent risk-based screening intervals [12]. As a proof-of-concept, we created a simplified organisational model for screening of individuals with IWGDF risk 0, based on individualised probability for LPP and LOPS (Fig. 2). The individual probability estimation needed for the organisational model was calculated as described above. The potential for biennial, rather than annual screening, for individuals stratified as IWGDF risk 0 and as having a low probability of complications (using 5% and 10% threshold), was estimated. The decision about the alternative biennial screening interval duration was arbitrary and was based on the local estimate of total numbers of screenings needed that would be manageable at our clinic.

Fig. 2
figure 2

Screening interval determination process

Full size image

Results

A total of 11,878 routine foot screenings including SWM, peripheral pulse palpation and foot examination were performed in individuals who regularly attended our centre between June 2015 and October 2020. A total of 10,948 records were eligible for inclusion after excluding incomplete datasets and records for individuals with a history of foot amputation at any level. The sample was further reduced due to incomplete diabetes data within our EHR (Fig. 1). The final complete sample for model generation consisted of 4282 records with 16 predictors (Tables 1, 2); there were no missing data in the final study sample. Age at screening was not statistically different (p = 0.38) between individuals who were either included in (65.06 ± 12.4 years) or excluded from (65.50 ± 13.3 years) the final sample. There was also no statistical difference (p = 0.14) in the proportion of male sex between the included (54.8%) and excluded (56.2%) group.

Table 2 Sample characteristics
Full size table

Using LOOCV, the penalised regression model based on all available predictors (16 parameters) yielded an AUC of 0.84 (95% CI 0.82, 0.85) for prediction of LOPS and AUC of 0.80 (95% CI 0.78, 0.83) for LPP. Logistic calibration analysis (using LOOCV) presented consistent recall of probabilities, with a Brier score of 0.08 (intercept 0.01 [95% CI −0.09, 0.12], slope 1.00 [95% CI 0.92, 1.09]) for LOPS and Brier score of 0.05 (intercept 0.01 [95% CI −0.12, 0.14], slope 1.09 [95% CI 0.95, 1.22]) for LPP. The model coefficients of penalised parameters are presented in the electronic supplementary material (ESM Tables 1–3, ESM Fig. 1), together with formulas for risk calculation (ESM Method 1, 2).

Histograms showing estimated probabilities for LOPS and LPP are presented in Fig. 3. Out of 4282 screening examinations, 3357 were performed on individuals without foot ulceration or Charcot foot history. Additionally, LOPS was detected in 533 (12.4%) and LPP in 251 (5.9%) individuals, leaving 3037 individuals who could be classified into IWGDF risk 0 group after screening. After calculating individual risk with leave-one-out generated regression prediction models, estimated probability for the presence of LOPS or LPP in the IWGDF risk group 0 was higher than 10% in 577 screenings (18.9%) and higher than 5% in 1426 screenings (47%) at the time of the investigation.

Fig. 3
figure 3

Histograms of estimated probabilities for LOPS and LPP

Full size image

In the subpopulation of individuals without deformities, LOPS was detected in 53 (4.8%) and LPP in 25 (2.3%) screening examinations. In 953 examinations performed on individuals with IWGDF risk 0 without deformities, the probability for LOPS or LPP was higher than 10% in 67 screenings (7%) and higher than 5% in 249 screenings (26%).

The calculated difference of probability from basal estimate, to up to 5 years after the screening, is presented in Fig. 4. Figure 4 shows relatively slow progression of individual probability of LOPS and LPP with time, when other clinical variables related to diabetes remained stable. Furthermore, the progression of probability seems clearly influenced by basal probability.

Fig. 4
figure 4

Estimated progress of probability for LOPS and LPP with time in individuals with stable clinical presentation of diabetes mellitus: difference from basal probability at the time of screening to estimated probability at subsequent years. Values with different probability at baseline are grouped according to the values displayed in the legend and represented with different colours. The lower and upper border of the boxes correspond to the first and third quartiles, the line in the middle indicates the median. The whisker extends from the box to the largest or smallest values no further than 1.5 × IQR (interquartile range). Values beyond the end of the whiskers are plotted individually as outlying points

Full size image

The existing recommendation for yearly screening of individuals with IWGDF risk 0 was modified to investigate the potential for increasing the interval between screening investigations (Fig. 2). The individuals in IWGDF risk 0 group with low probability of LOPS or LPP would be scheduled for screening at 2-year intervals instead of 1 year. In a hypothetical follow-up of 2 years, we demonstrated a 40.5% reduction in the absolute number of screening examinations in individuals with IWGDF risk 0 (3614 instead of 6074 screenings) when a 10% probability cut-off was used and a 26.5% reduction (4463 instead of 6074 screenings) when the risk cut-off was set to 5%. Further reductions were observed when the proposed model was applied to the population of individuals without deformities (36.9% at 5% probability cut-off; 46.8% at 10% probability cut-off).

Discussion

Clinical prediction models have been used in screening for diabetes in the general population [19] and for automated staging of diabetic retinopathy [20]. Recent studies of machine learning methods show promising results in the classifications of diabetic foot ulcers [21], determination of risk for ulceration on the foot based on genetic risk [22], prediction of healing of diabetic foot ulcerations [23] and for diagnostic of diabetic neuropathy based on pictures of corneal confocal microscopy [24]. The screening interval is increasingly recognised as a public health blind spot, and offers many opportunities for improvement of healthcare [12]. While there is convincing evidence for clear safety and cost-effectiveness of using individualised screening intervals for retinopathy detection [25], similar studies for customisation of diabetic foot screening is lacking.

Main findings

We have developed a prediction model for diabetic foot complications that shows good discrimination ability with a cross-validated AUC of 0.84 for LOPS and 0.80 for LPP. Furthermore, we could demonstrate a consistent recall of probabilities according to the calibration analysis. Calculation of probabilities for LOPS and LPP in individuals with diabetes and stage 0 IWGDF risk in our centre shows highly imbalanced risk distribution. The estimate of probability distribution of risk was generated on the population of individuals attending a tertiary diabetes clinic, so the imbalance may be even larger when investigated in a population comprising mostly of those at IWGDF risk stage 0 who usually attend primary care centres in the early stages of diabetes. Most foot screenings not unsurprisingly are performed in individuals at low risk (Fig. 3). Additionally, the risk for foot complications over years seems to increase more slowly in people with low baseline risk than in their high-baseline-risk counterparts (Fig. 4). Therefore, identification of the low-risk subpopulation presents an opportunity for optimising the screening process.

Improvement of the existing screening model

We have created a simplified theoretical organisational model in order to improve current diabetic foot screening practice (Fig. 2) and tested it on our clinic sample. The proposed approach is in line with the recent consensus of ADA and EASD on precision medicine in diabetes, stressing that the practice of medicine should centre on the individual and his/her unique characteristics [26]. The results of our analysis suggest the possibility of a substantial reduction in the number of diabetic foot screenings. This could optimise screening intervals based on individual probability for observed complications at the time of screening. In this proof-of-concept study, we demonstrated a 26.5% (cut-off, 5% risk) to 40.5% (cut-off, 10% risk) reduction in the absolute number of screenings needed for a population with IWGDF risk 0 in a 2-year period when the screening interval was extended to biennially instead of annually in the low-risk group. The reduction was even more pronounced when the model was tested in the IWGDF risk 0 group without foot deformities. Our findings are similar to those of Broadbent et al. [25], who evaluated a model for individualised retinopathy screening and observed a 43.2% reduction in the number of diabetic retinopathy screening appointments in the individualised arm of their study when compared with a fixed screening interval.

Implications for practice

Introduction of systematic diabetic foot care frameworks clearly improves many outcomes [27] that have often not been paired with adequate reimbursement, even though increasing evidence shows that the long-term costs for implementation can be offset by improved access to care as well as by reduced foot complications and amputation rates [28].

With rising numbers of people with diabetes, adopting the screening rates proposed by the guidelines would be associated with substantial financial and organisational burden for the healthcare system. This burden is often perceived as an obstacle, from the decision makers’ point of view, to guideline implementation. Additional funds for improvements in quality and capacity of care are often approved only within an existing framework because of finite resources within healthcare systems [29]. Therefore, improvements that enable better efficiency are more likely to be implemented than those that require extension off existing capacities.

Currently, the patients with diabetes at our centre are not screened according to the guidelines, predominantly due to insufficient capacity and only partial insurance reimbursement. Introduction of the framework that proposes fewer examinations than currently advised by guidelines would paradoxically increase the absolute numbers of screenings performed in individuals with IWGDF risk 0 at our centre, thereby improving safety and efficiency. The proposed screening model assumes a stable clinical situation without worsening of complications or metabolic control, which in reality frequently occurs. The screening interval should therefore be shortened if new symptoms arise or if significant worsening of metabolic control is detected during regular clinic visits.

Strengths and weaknesses of the study

A large number of predictors would be very difficult to implement in traditional care, but the era of wider use of structured EHR databases and availability of automated computation enable much broader application of clinical prediction models to practitioners at the point of care. The use of software-assisted documentation methods may bring additional advantages for foot clinics with low resources, in comparison with paper-based methods, in early diabetic foot complications detection [30]. Eleven routine variables that we have selected as parameters for our model are often already present in structured digital form in current local computer systems. Our dataset extends routine data with only five diabetic foot screening variables (in addition to LOPS or LPP) that are routinely collected for risk staging as proposed in current foot screening guidelines and thus would already be documented.

The limitations stem from the design of the study and the lack of validation of the proposed model in prospective studies and other populations. Improved results may be obtained by including additional clinical variables associated with progressing diabetes complications and with creation of prediction models based on different types of diabetes [31]. Additional model-building methods should be compared and optimised. Even in the relatively large sample size used in our study, there is a possibility of bias from insufficient number of events relative to the number of predictors when considering LPP (14.8 events per predictor) [32]. The exclusion of people with amputations may influence external validity, although this only corresponded to 7% of the screened population.

The proposed prediction model targets surrogate markers for neurological and vascular damage that are used for screening (LOPS and LPP). Further studies could improve discrimination ability by targeting more specific and direct outcomes of foot complications over time. While prediction models should ideally be based on longitudinal cohorts, with estimation of real future events, our approach also offers some advantages. Existing prospective clinical trials often do not include all groups from the wide spectrum of population subtypes. These need to be covered in prediction models for wider population screening. Models generated from local clinical databases can be generated much more easily and less expensively than large longitudinal cohorts. Additionally, they would probably more accurately reflect specific properties of the local population to which the screening will be applied.

Unanswered questions and future research

The favourable characteristics of the generated model warrant further study to improve this model. Our study was not designed to answer the question about optimal screening interval duration in different risk categories because of the limitations of our database. Additional studies on data from people with more frequent foot examinations within an observed period are needed to explore optimal screening interval length. Studies are also required regarding safety and optimal inclusion of this model into different practical screening scenarios with concomitant selection of screening parameters.

The proposed model could be further improved with additional parameters from automated output (e.g. by current emerging tele-health possibilities in diabetic foot care [33] and patient-reported outcome measures [34]). This could widen the routinely collected clinical dataset used in the current model. Additionally, the model could be further extended by inclusion into multi-modal screening programmes with an even more extensive dataset generated at the time of screening [35] and by the use of mobile devices [36] or genetic markers [22].

Our model is not easily applicable to settings without EHR systems, which unfortunately is still the reality of several developing countries. There are ongoing efforts to help healthcare institutions without existing EHR systems to introduce optimised forms of digital documentation. One such initiative, specific for diabetic foot, is DIAFI-DATA [37], which was developed for environments with limited resources and can be used out of a single USB flash drive and without network connectivity. The main benefit of penalised regression methods is in their capacity for selecting a subset of indicators within an often substantially larger pool of available predictors (especially least absolute shrinkage and selection operator [LASSO]). Thus, they should be further explored for selection of a smaller and more manageable set of prediction parameters that could be used in clinical practice with or without EHR systems in place [15].

Conclusions

The diabetes-related burden for healthcare systems is increasing with ageing populations, and increasing diabetes prevalence and obesity worldwide. Improvements in healthcare systems are urgently needed to enable people with diabetes to receive adequate care, which is often not optimal because of the lack of systemic resources.

A protocol for diabetic foot screening that is enhanced by a risk prediction model enables stratification of individuals based on the likelihood of complications. Discrimination of low ulcer risk could extend the frequency of screening for those at low risk, thus freeing up resources for those at higher risk of foot ulcer. The proposed model requires further refinement and external validation, but it shows potential for improving compliance with guidelines for screening, which are often not appropriate.