Validity and reliability of smartphone use in assessing balance in patients with chronic ankle instability and healthy volunteers: A cross-sectional study

Highlights

• “MyAnkle” application is valid for testing balance when eyes are closed.

• It can be used to assess balance in patients with chronic ankle instability.

• It is valid in assessing balance in healthy volunteers.

• Its test-retest reliability within one-week ranges from poor to moderate.

• The application scores do not distinguish patients and healthy volunteers.

Abstract

Background

Chronic ankle instability (CAI) is associated with defective posture control and balance; thus, a proper assessment of these impairments is necessary for effective clinical decision-making. There is a need for portable, valid, and reliable methods to facilitate the easy collection of real-world data, such as mobile phones.

Research question: Is the smartphone “MyAnkle” application valid and reliable in assessing balance in patients with CAI and healthy volunteers?

Methods

This was a cross-sectional study. Sixty-five participants completed two assessment sessions, including 31 patients (n = 41 ankles with CAI and 21 asymptomatic ankles) and 34 healthy volunteers (n = 68 ankles). In each session, dynamic single-leg stance balance was measured simultaneously using the "MyAnkle" application and the Biodex balance system (BBS) version 3. Testing was conducted at three levels of BBS difficulty—4 (D4, hard, loose platform), 6 (D6, moderate), and 8 (D8, easy, stiffer platform)—and repeated with opened and closed eyes. Both limbs were tested in a random order by two independent blinded assessors.

Results

The two devices showed significant poor-to-moderate correlations when eyes were closed (p < 0.05). For discriminant validity, the application did not distinguish the two study groups in all tested conditions (p > 0.05), whereas the BBS weakly to moderately distinguished the dominant limbs in the two groups at all difficulty levels with eyes-open and at D8 with eyes-closed regardless to limb dominance. For reliability, a significantly poor to moderate inter-session reliability was noted for the two devices.

Significance

"MyAnkle" application is valid in assessing balance in patients with CAI when the eyes are closed. However, similarly to BBS, its one-week test-retest reliability may be insufficient for accurate follow-up of balance changes and need to be interpreted with caution. Future studies need to establish its inter-tester reliability and its usefulness in telerehabilitation.

Introduction

Lateral ankle sprain is a highly prevalent and recurring musculoskeletal injury in adults [1,2]. Its secondary impairments may persist, leading to chronic ankle instability (CAI) [3]. This may impair physical activity, and alter lower-limb mechanics, postural control and balance [[4], [5], [6]].

The assessment of these impairments is therefore essential to establish an effective rehabilitation plan and to prevent recurrence. There are several methods to assess balance such as physical functional performance tests [7] and the Biodex balance system (BBS) [8,9]. However, most of these tests conduct requires a trained assessor and/or special testing settings or equipment. Thus, they are difficult to incorporate in telerehabilitation or to gather real-world data. Therefore, there is a need for portable, valid, and reliable tools, that do not need a skilled professional to operate, in order to assess patients at different clinical and non-clinical settings.

Wearable sensors, including mobile health applications, may be efficient, accurate and reliable in assessing and training balance indoors and outdoors [[10], [11], [12]]. Further, they are suitable for telerehabilitation as they allow wireless data transfer. Many smartphone applications are available to evaluate balance and falling risk in stroke survivors and frail elderly individuals [[13], [14], [15]]. Mobile applications were also used to assess balance in healthy people and participants with CAI; however, they were not validated against reference standard methods [[16], [17], [18]]. “MyAnkle” is a smartphone application that was developed to assess standing balance in healthy volunteers. This application was validated against subjective experts’ rating rather than using objective methods. Thus, evidence of its concurrent validity is not robust to recommend its clinical use. Further, its inter-session test-retest reliability has not been established yet [16], which is essential to enable clinicians from assessing and communicating patients’ progress overtime. Therefore, the primary purpose of this study was to investigate the concurrent validity of the "MyAnkle" application in assessing balance in patients with CAI and healthy volunteers. Secondarily, this study assessed the application’s discriminant validity and inter-session test–retest reliability. We hypothesized that the applications’ scores would: (1) at least moderately correlate with that of the BBS; (2) show at least moderate test-retest reliability; and (3) distinguish between patients with CAI and healthy volunteers.

This was a cross-sectional study that was conducted over two sessions at the Biodex balance laboratory at the Faculty of Physical Therapy, Cairo University, Egypt, between July 2018 and May 2019. The study was approved by the institutional ethics committee (P.T.REC/012/001991) and registered at ClinicalTrials.gov (NCT035989850).

Sixty-seven consecutive participants were enrolled in this study, including 33 patients (21 with unilateral CAI and 12 with bilateral CAI; with a total of 45 unstable ankles) and 34 healthy volunteers (68 ankles). A minimum of 29 ankles per group was required based on sample size calculation using a priori sample consisting of 20 ankles from 10 patients with unilateral CAI and 20 asymptomatic ankles from 10 healthy volunteers. Both limbs under different testing difficulty levels and eye conditions were used to calculate the sample size; with an α error set at 0.05, a statistical power of 0.80, and an estimated correlation strength of 0.5 (http://www.sample-size.net/correlation-sample-size/).

Individuals were recruited from the outpatient clinic of the faculty of Physical Therapy,Cairo University, based on the following inclusion criteria: age ranging between 18 and 35 years old [16], a confirmed diagnosis of CAI, a Cumberland Ankle Instability tool (CAIT) score <27 points [19], and having experienced at least one recurrent sprain within the past year. Patients were excluded if they were overweight, had a history of lower quadrant fracture or surgery (past two years), lower-extremity injury (past three months), or other causes of defective balance (past three months), participated in balance training, or had lower limbs muscle weakness confirmed during manual screen testing. Healthy volunteers were enrolled from college students based on the same eligibility criteria, except that they were free from previous ankle pain or injury.

Participants were screened against study eligibility criteria. CAI diagnosis was confirmed by the first author (a physiotherapy clinical instructor with a 5-year experience). After a full verbal explanation of the testing procedures, participants who agreed to participate were directed to sign an informed consent. Then, basic demographic data were collected.

Each participant was assessed in two separate sessions with a one-week interval in between. During each session, balance was simultaneously measured using the “MyAnkle” application (http://www.eecg.utoronto.ca/∼jayar/CAM/myankle-2.html) and the BBS version 3 (Biodex Medical Systems, Inc., Shirley, NY, USA). The application was installed on a HUAWEI P9 lite version 5 android (HUAWEI VNS-L31) smartphone.

The application assesses balance by analyzing the data derived from the smartphone’s built-in accelerometer. Accelerometers quantify the magnitude of acceleration in the X, Y, and Z axes that correspond to movements in the Mediolateral (ML), cephalo–caudal, and Antero-posterior (AP) directions, respectively. Values obtained from each accelerometer are initially corrected for static bias; by dividing the gravitational acceleration by the mean of all the samples in a given axis. Then, the magnitude of the resultant vector (R) is calculated by summing the corrected values squared in all axes. The average R value represents the overall balance index score [16].

The BBS was used as a reference tool. It has a sampling rate of 20 HZ, and measures stability indices in AP and ML directions only and, accordingly, calculates an overall balance index score. This score represents the variance of the platform displacement (degrees) from the level position [20], and consequently, the center of Pressure (CoP) excursion. The BBS has adequate inter- and intratester reliability [intraclass correlation coefficient (ICC): 0.42–0.80] [8,20]. In this study, overall balance scores of the two devices were compared. Before testing, the two devices were calibrated according to manufacturing guidelines.

Each participant had their two limbs tested in a random order generated by the Microsoft Excel software’s random function. Both limbs were assessed to account for the possible effect of limb-dominance [21].

Initially, the smartphone was secured above the superior midline of the patella using an adjustable armband [16] (supplementary Fig. 1). Participants were then asked to assume a single-leg stance (SLS) on the locked BBS platform. Afterward, the platform was released and the application was initiated. During the testing process, participants were asked to keep the cursor on the BBS screen in the center during platform perturbation without moving their foot off the platform. Participants were allowed two minutes of rest between the testing of their two extremities [8].

For validity testing, patients were tested at three levels of balance difficulties—8 (D8, easy), 6 (D6, moderate), and 4 (D4, hard)—starting from the easiest to the hardest. Difficulty level is determined based on the platform tilting angle from the level position, which varies from zero to 20°. Increased tilting angle requires higher balance skills to restore the platform level, and the reverse is true [20]. Participants were asked to keep their hands on their hips throughout all testing without performing any disqualifying compensatory motions [8]. Testing was conducted involving two eye conditions: (1) opened (for 30 s) and (2) closed eyes (for 10 s). Testing with eyes-closed aimed at isolating the cognitive effect of vision. For each testing level, participants received a training trial prior to actual data collection. For inter-session test–retest reliability, the same exact testing procedures were repeated one week later.

Participants’ preparation and BBS testing were overseen by a trained assessor (N. A.) using a standard protocol. A second assessor (B. A.), blinded to BBS scores, participants’ grouping, and limb conditions, managed the application throughout the testing procedures. The two assessors were instructed not to communicate together regarding the recorded scores. Further, data were not entered into the Excel sheet until data collection from the two testing sessions was complete, to maintain blinding to previous recordings.

MyAnkle application and BBS scores were tabulated in a Microsoft Excel spread sheet. For demographic categorical variables, the chi-square test was used to compare between the groups. Quantitative data, first, were screened for normality using the Shapiro–Wilk test, and accordingly, the nonparametric Mann–Whitney U test was employed to compare participants’ baseline characteristics. Descriptive statistics are presented as medians and ranges.

To assess concurrent validity, Spearman rank correlation coefficients (ρ) were calculated to examine the strength of association between the balance scores of the two devices under the two eye conditions, for the two limbs in the dominant/nondominant and injured/least- or noninjured conditions, for healthy volunteers and patients, respectively. Correlation results were interpreted as negligible (ρ < 0.30), weak (ρ = 0.30–0.50), moderate (ρ = 0.50–0.70), high (ρ = 0.70–0.90), or very high (ρ > 0.90) [22].

Discriminant validity was examined using the Mann–Whitney U test to compare the balance scores between the two groups, based on limb dominance. Further, effect size was calculated according to Wendt [23] and interpreted as explained earlier for correlation results.

To assess test–retest inter-session reliability, the ICC was used to examine the agreement between the BSS and “MyAnkle” scores in each tested group using two-way mixed effects model (absolute agreement, single measurement). Agreement strength was interpreted as excellent (> 0.90), good (0.75–0.90), moderate (0.50–0.75), or poor (< 0.50). Standard Error of Measurement agreement (SEM_agreement) was calculated by taking the square root of the error variance of an ANOVA analysis (√within people residual mean square).The minimal detectable change (MDC) for the two instruments was calculated using the formula 2.77 x SEM at 95% confidence level [24]. MDC represents the value that can be considered real change above measurement error.

The level of significance was set at p < 0.05 throughout all analyses. All statistical analyses were conducted using the SPSS version 21 for Windows (IBM Corp., Armonk, NY, USA).