http://orcid.org/0000-0003-0697-3703
Figures
Abstract
Objectives
Although more than 1.8 million people survive snakebite envenomation each year, their recovery is understudied. Obtaining long-term follow-up is challenging in both high- and low-resource settings. The Patient-Specific Functional Scale (PSFS) is an easily administered, well-accepted patient-reported outcome that is validated for assessing limb recovery from snakebite envenomation. We studied whether the PSFS is valid and reliable when administered by telephone.
Methods
This is a secondary analysis of data from a randomized clinical trial. We analyzed the results of PSFS collected in-person on days 3, 7, 14, 21, and 28 and by telephone on days 10, 17, and 24. We assessed the following scale psychometric properties: (a) content validity (ceiling and floor effects), (b) internal structure and consistency (Cronbach’s alpha), and (c) temporal and external validity using Intraclass Correlation Coefficient (ICC). Temporal stability was assessed using Spearman’s correlation coefficient and agreement between adjacent in-person and telephonic assessments with Cohen’s kappa. Bland Altman analysis was used to assess differential bias in low and high score results.
Results
Data from 74 patients were available for analysis. Floor effects were seen in the early post-injury time points (median: 3 (IQR: 0, 5) at 3 days post-enrollment) and ceiling effects in the late time points (median: 9 (IQR: 8, 10). Internal consistency was good to excellent with both in-person (Cronbach α: 0.91 (95%CI 0.88, 0.95)) and telephone administration (0.81 (0.73, 0.89). Temporal stability was also good (ICC: 0.83 (0.72, 0.89) in-person, 0.80 (0.68, 0.88) telephone). A strong linear correlation was found between in-person and telephone administration (Spearman’s ρ: 0.83 (CI: 0.78, 0.84), consistency was assessed as excellent (Cohen’s κ 0.81 (CI: 0.78, 0.84), and Bland Altman analysis showed no systematic bias.
Author summary
Snakebite envenomation is an important but neglected tropical disease that impacts millions of people worldwide each year. These bites lead to both death and permanent disability. As they occur in tropical and subtropical regions, they primarily impact people from low-income areas of the world. As potential new treatments are being developed, we must understand their potential benefit in humans before they can be widely disseminated. Performing these human studies requires the ability to determine how patients recovered with these treatments. Having people return for evaluation during recovery is difficult in these low-income regions. We evaluated the ability to use a telephone version of an already accepted measurement of recovery in snakebite, the Patient-Specific Functional Scale. This study demonstrates that using this telephone-administered measure is feasible, valid, and reliable. With the results of this study, we now have an important tool to easily measure recovery in areas where snakebite predominates. This tool will help snakebite envenomation researchers evaluate the potential benefit of new treatments and accelerate the process of bringing new effective treatments to those snakebite patients in the most need.
Citation: Theophanous RG, Vissoci JRN, Wen FH, Griffin SM, Anderson VE, Mullins ME, et al. (2019) Validity and reliability of telephone administration of the patient-specific functional scale for the assessment of recovery from snakebite envenomation. PLoS Negl Trop Dis 13(12): e0007935. https://doi.org/10.1371/journal.pntd.0007935
Editor: Janaka de Silva, Faculty of Medicine, University of Kelaniya, SRI LANKA
Received: August 21, 2019; Accepted: November 19, 2019; Published: December 13, 2019
Copyright: © 2019 Theophanous et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The parent studies, from which data were derived, were funded by BTG International Inc. There was no specific funding for this substudy. The funder of the parent study had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: I have read the journal's policy and the authors of this manuscript have the following competing interests: Dr. Rose reports speakers bureau fees from BTG International Inc outside the submitted work. Dr. Greene reports receiving consulting fees and honoraria from BTGInternational Inc outside the submitted work. Drs. Gerardo, Charlton, Mullins, Lavonas and Kleinschmidt and Ms. Anderson report receiving prior study-related grants from BTG International Inc.
Introduction
Snakebite envenomation is a neglected tropical disease that affects as many as 1.8 million people per year with the overwhelming majority of patients from low- and middle-income countries (LMICs). Although snakebite envenomation is responsible for an estimated 94,000 deaths annually, the burden of injury is also immense, as many of the survivors sustain permanent disability.[1–5] To date, almost no clinical trials have attempted to study the impact of treatment interventions on snakebite-caused disability.[6–10] However, researchers face substantial challenges to performing high quality trials, and research instruments used to assess disability and recovery must be both validated and practical to administer in low-resource settings.
An essential element of high-quality clinical research is the use of patient-centered outcome measures, such as patient reported outcomes (PROs). Currently, no practical, inexpensive, reliable, validated PROs exist that are appropriate for evaluating patients with snakebite envenomation.[11, 12] This impacts snakebite envenomation research, particularly in LMICs due to cost and logistical barriers to in-person administration of a PRO. The patient may need to take time off from work, pay for transportation, coordinate childcare, or navigate the innumerable barriers that already exist to access healthcare in order to participate in an in-person outcome assessment. The ability to use a valid, reliable outcome measure administered by telephone eliminates many of these challenges. With the widespread use of cellphones in LMICs, a telephone-administered, validated PRO would be an inexpensive and useful tool in future snakebite envenomation research. [13]
The Patient-Specific Functional Scale (PSFS) is a validated, patient-centered measurement tool that assesses a patient’s functional impairment regarding specific physical activities that the patient identifies as important. Patients report three to five activities or tasks that they are unable to perform or have difficulty with due to their illness.[14] The validity of the PSFS has been demonstrated in numerous studies, particularly in those related to musculoskeletal disease or injury.[15–19] The PSFS administered in person has also been validated in studies involving patients with snakebite envenomation to the extremities.[11, 20] In fact, the PSFS administered in-person is highly correlated with more complex assessment tools and is very responsive to changes in patient functional status over time.[20] The PSFS can logistically be performed by telephone, but no studies have validated telephone-administered PSFS compared to in-person administration.
A recent snakebite envenomation clinical trial used in-person PSFS as the primary outcome and recorded additional assessments with telephone administered PSFS.[21] This provides the opportunity to validate the telephone version against the in-person criterion standard. The purpose of our study is to determine if telephone administered PSFS has similar validity, reliability, and consistency to in-person administration in this snakebite envenomation population. We report the psychometric properties of telephone compared to the PSFS administered in-person, specifically looking at: (a) content validity, (b) internal structure and consistency, and (c) temporal stability and external validity.
Methods
Ethics statement
The current study is a secondary analysis. The parent study procedures were reviewed and approved by the Western Institutional Review Board (WIRB) and by the IRB responsible for each clinical site. The activities of the coordinating center were approved by the Colorado Multiple IRB (COMIRB). Written informed consent was obtained from all subjects prior to participation.
Study design
We performed a secondary analysis of the “The efficacy of Fab antivenom versus placebo plus optional rescue therapy on recovery from copperhead snakebite envenomation: a randomized, double-blind, placebo-controlled clinical trial.” The complete methods of this trial and participant selection have been published previously, but are detailed in brief. [21]
Patient selection
Patients age 12 years or older with mild or moderate copperhead envenomation were randomized to receive Fab antivenom (FabAV) or placebo. Patients with severe envenomation, envenomation proximal to the elbow or knee, more than one extremity involved, pregnancy, or presenting >24 hours after envenomation were excluded. All 72 patients from the modified intention-to-treat population of this study were included in the analysis.
Instrument
The original scale used was the PSFS, which assesses a patient’s ability to perform an important physical activity or task and consists of three to five activities in a single dimension. As the performance of the three-activity and five-activity tools are similar, the three-activity PSFS was chosen for this trial. The answers are given on an 11-point scale (0 = Unable to perform activity to 10 = Able to perform activity at the same level as before the injury or problem). Higher values indicate improved function. At the initial assessment, patients were asked to list three specific activities of their choosing and rank their ability to perform them. They were asked the same questions at follow up assessments at particular time intervals to monitor if their physical function was still affected and to what degree.[14] The questionnaire served as a tool to quantify activity limitation and measure functionality.
Data collection
In-person PSFS was performed on days 3, 7, 14, 21, and 28 post-envenomation (+/- 1 day). Telephone PSFS was performed on days 10, 17, 24, and >28 post envenomation (+/- 1). As the in-person and telephone PSFS were not obtained on the exact same day, we use the following nomenclature: T1 (days 7 and 10), T2 (days 14 and 17), and T3 (days 21 and 24). An independent Data Monitoring Committee oversaw study conduct, and on-site monitoring was performed by the sponsor’s clinical research associates.
Analysis
We first compared the descriptive mean scores between in-person and telephone administration of the PSFS using a t-test comparison, including calculations for standard deviation. We then compared telephone to in-person administration of PSFS by evaluating the following scale psychometric properties: content validity, reliability, and external validity.
We assessed content validity by evaluating the proportion of patients with floor and ceiling effects over time. We assessed reliability by evaluating internal consistency, temporal stability, and association between instruments. Regarding internal consistency, we used Cronbach’s alpha to determine if in-person and telephone-administered PSFS produced similar results. We considered a Cronbach’s alpha value above 0.7 as good.[22] We then measured temporal stability to assess the instrument's variation in time and to verify the test–retest reliability of the instrument, with an Intraclass Correlation Coefficient (ICC) above 0.75 considered adequate.[23] To evaluate temporal stability using times when the patient would be expected to have achieved a stable state of recovery. [21] Thus, we used day 14 to day 21 for in-person administration (T2 to T3) and day 17 to day 24 for the telephone measurements (T2 to T3) to calculate the ICC.
We used a Bland-Altman analysis to assess agreement between in-person and telephone administered PSFS versions, plotting the difference between two paired measurements against the average of the two measurements. The resultant values were plotted, with a line indicating perfect agreement (i.e. zero difference), allowing a visual assessment of variation related to the size of the mean. Confidence intervals (i.e. limits of agreement) were also calculated using non-parametric methods. [24, 25] We assessed external validity of the telephone-administered PSFS scores in comparison to the in-person–administered PSFS, using Spearman correlation coefficients (ρ).
Results
Our sample was composed of 72 adults from 13 centers. Fifty-two percent were male and 89% were over 18 years of age. The mean age was 43 (SD 17.6) years. A tendency toward floor effects could be seen in the total PSFS with a median score of 3 (interquartile range (IQR): 0–5; range 0–8). On the initial (day 3) assessment, 28% of the participants scored 0, indicating a severe activity limitation. Ceiling effects were more evident on follow-up days 24 and 28, as the median score of PSFS total was 9 (IQR: 8–10; range 5–10), and 48% of the participants scored 10, which indicated no self-reported limitation in the chosen activities.
Internal consistency was good to excellent
Cronbach alpha values were above 0.9 (excellent) for in-person PSFS and between 0.7 to 0.9 (good) for telephone PSFS for all parameters (Table 1). The ICC confirmed good temporal stability of the scale, with ICC 0.83 (95% CI 0.72, 0.89) in-person and 0.80 (95% CI 0.68, 0.88) via telephone. Rating differences between in-person and telephone administered measurements ranged from -8 to 2, with a median difference of 0. Bland-Altman analysis found that 95% of the differences between paired ratings were between -4 and +1, with 50% of the differences between -2 and 0 (Fig 1). Overall there were only 4 sets of measures (5%) with differences of at least 6 points.
The Spearman Correlation coefficient demonstrated a very strong linear correlation (>0.8) between PSFS administered in-person and telephone (ρ: 0.83 (95% CI 0.80, 0.93) (Fig 2). Comparison of the in-person and telephone administered PSFS over time shows a time dependent progression of PSFS regardless of method of administration (Table 1).
Discussion
Based on multiple analyses, we determined that results obtained from in-person and telephone administrations of the PSFS are equivalent. Both scales exhibit a floor effect at earlier time points and a ceiling effect at later time points, signifying that patients who sustain copperhead snakebites have significant early functional disability and a good long-term recovery response. Both scales have acceptable (good to excellent) internal consistency and good temporal stability. ICC and correlation coefficient, which are a measure of between–scale reliability, also demonstrates very good association.
Telephone-based testing has been previously used as an alternative data collection method in clinical research. The use of telephone versus in-person PROs has been studied[26, 27], however, this is the first study looking into the PSFS as a telephone-administered PRO. These results are important as the PSFS has uniquely flexible characteristics that have not yet been evaluated with different administration methods. Our results suggest that the PSFS may function well in alternative methods of administration such as text messaging, online surveys, and mobile applications.
The internal consistency of the telephone administered PSFS in our study was excellent and comparable to the in-person version. Reliability of the telephone administered PSFS in snakebite envenomation is similar to the findings of prior studies performed using in-person PSFS in knee dysfunction, acute and chronic low back pain, cervical pain, chronic lateral epicondylitis, and chronic obstructive pulmonary disease (COPD).[14, 28–31]
The telephone administered PSFS demonstrated good stability over time, comparable to the in-person PSFS in snakebite envenomation. Additionally, its temporal stability is similar or better than the in-person PSFS in other injuries or diseases.[19] The existing literature has reported test–retest values ranging from 0.55–0.95.[16, 19, 30] These findings are concordant with the Bland Altman analysis which suggests that the telephone-administered PSFS is a reliable tool with minimal bias.
The external validity of the telephone administered PSFS against the in-person PSFS was also high and consistent with the literature on other PROs administered by telephone.[27, 32] However, the PSFS has the advantage of flexibility in patient choice of activities assessed. This adaptability of the instrument allows the outcome to be tailored to an individual patient and assesses function across a diverse population. In contrast to many other PROs that would be reasonable to use in snakebite research, the PSFS is not limb-specific. This allows for assessment of a wide range of patients, including those with upper extremity and lower extremity bites, with results that are broadly generalizable in snakebite envenomation.
Based on our results, we now have a PRO that is easily used in clinical research where in-person measurements are financially and logistically prohibitive. This is the case in global snakebite envenomation research, particularly in LMICs where this disease predominates.[5, 10] The PSFS is the most extensively validated PRO in snakebite envenomation research, and telephone administration makes this instrument pragmatic to use in real-world settings.[11, 12, 20] Given the potential use of the telephone administered PSFS, researchers should be aware that mobile phone penetrance is not universal, but its use has been recommended by the World Health Organization and the National Institutes of Health Fogarty Center [33–35]. Additionally, there are communities where information by phone could be biased due to the presence of authoritarian regimes, political violence or organized crime. Lastly, the telephone PSFS measures functionality which is one of the most important outcomes in snakebite envenomation. However, it does not measure other important patient-centered outcomes such as psychological sequela, disfigurement leading to stigmatizations, or others.
Limitations
Our study has several limitations that could impact the interpretation of our results. First, this is a secondary analysis of an existing data set and not the primary hypothesis of the original study. Therefore, we could not directly compare the telephone to in-person PSFS administration at the identical time points as we were limited to the original data collected. Nevertheless, because our times points are very close to each other, the curves for PSFS administered in-person and by telephone overlap substantially and thus are very similar. Additionally, our study is specific to copperhead envenomation, which on average has less severe envenomation than other snakebites such as rattlesnakes or tropical venomous snakes.[36, 37] In generalizing our results to other snake species, there is risk that the early floor effects would be greater. However, these floor effects disappear over time during recovery. It is likely that these floor effects would also resolve with envenomation from other species during recovery, making the tool useful for assessing recovery at greater than 3 days.
Supporting information
S1 Dataset. Patient-specific functional scale scores reported in-person on days 3, 7, 14, 21, and 28 post-envenomation (+/- 1 day) and by telephone on days 10, 17, 24, and >28 post envenomation (+/- 1).
https://doi.org/10.1371/journal.pntd.0007935.s001
(CSV)
References
- 1. Williams D, Gutierrez JM, Harrison R, et al. The Global Snakebite Bite Initiative: an antidote for Snakebite bite. Lancet. 2010;375(9708):89–91. pmid:20109867
- 2. Lancet The. Snakebite-bite envenoming: a priority neglected tropical disease. Lancet. 2017;390(10089):2.
- 3. Kasturiratne A, Wickremasinghe AR, de Silva N, et al. The global burden of Snakebitebite: a literature analysis and modelling based on regional estimates of envenoming and deaths. PLoS Med. 2008;5(11):e218. pmid:18986210
- 4. Harrison RA, Hargreaves A, Wagstaff SC, Faragher B, Lalloo DG. Snakebite envenoming: a disease of poverty. PLoS Negl Trop Dis. 2009;3(12):e569. pmid:20027216
- 5. Harrison RA, Gutierrez JM. Priority actions and progress to substantially and sustainably reduce the mortality, morbidity and socioeconomic burden of tropical Snakebite. Toxins (Basel). 2016;8(12):e351.
- 6. Lewin M, Samuel S, Merkel J, Bickler P. Varespladib (LY315920) appears to be a potent, broad-spectrum, inhibitor of Snakebite venom phospholipase A2 and a possible pre-referral treatment for envenomation. Toxins (Basel). 2016;8(9):e248.
- 7. Komives CF, Sanchez EE, Rathore AS, et al. Opossum peptide that can neutralize rattlesnake venom is expressed in Escherichia coli. Biotechnol Prog. 2017;33(1):81–6. pmid:27718338
- 8. Renu K, Gopi K, Jayaraman G. Formulation and characterisation of antibody-conjugated soy protein nanoparticles—implications for neutralisation of snakebite venom with improved efficiency. Appl Biochem Biotechnol. 2014;174(7):2557–70. pmid:25185504
- 9. Chen YJ, Tsai CY, Hu WP, Chang LS. DNA Aptamers against Taiwan banded krait alpha-bungarotoxin recognize Taiwan cobra cardiotoxins. Toxins (Basel). 2016;8(3):e66.
- 10. Williams DJ, Faiz MA, Abela-Ridder B, et al. Strategy for a globally coordinated response to a priority neglected tropical disease: snakebite envenoming. PLoS Negl Trop Dis. 2019;13(2):e0007059. pmid:30789906
- 11. Gerardo CJ, Vissoci JRN, de Oliveira LP, et al. The validity, reliability and minimal clinically important difference of the patient specific functional scale in Snakebite envenomation. PLoS One. 2019;14(3):e0213077. pmid:30835744
- 12. Lavonas EJ, Gerardo CJ, Copperhead Snakebite Recovery Outcome Group. Prospective study of recovery from copperhead Snakebite envenomation: an observational study. BMC Emerg Med. 2015;15:9. pmid:25975429
- 13. Arie S. Can mobile phones transform healthcare in low and middle income countries? BMJ. 2015;350:h1975. pmid:25902967
- 14. Stratford P, Gill C, Westaway M, Binkley J. Assessing disability and change on individual patients: a report of a patient specific measure. Physioth Can. 1995;47(4):258–63.
- 15. Pengel LH, Refshauge KM, Maher CG. Responsiveness of pain, disability, and physical impairment outcomes in patients with low back pain. Spine (Phila Pa 1976). 2004;29(8):879–83.
- 16. Cleland JA, Fritz JM, Whitman JM, Palmer JA. The reliability and construct validity of the Neck Disability Index and patient specific functional scale in patients with cervical radiculopathy. Spine (Phila Pa 1976). 2006;31(5):598–602.
- 17. Koehorst ML, van Trijffel E, Lindeboom R. Evaluative measurement properties of the patient-specific functional scale for primary shoulder complaints in physical therapy practice. J Orthop Sports Phys Ther. 2014;44(8):595–603. pmid:25029915
- 18. Novak CB, Williams MM, Conaty K. Evaluation of the patient-specific functional scale in hand fractures and dislocations. Hand (N Y). 2015;10(1):85–7.
- 19. Horn KK, Jennings S, Richardson G, Vliet DV, Hefford C, Abbott JH. The patient-specific functional scale: psychometrics, clinimetrics, and application as a clinical outcome measure. J Orthop Sports Phys Ther. 2012;42(1):30–42. pmid:22031594
- 20. Anderson VE, Burnham RI, Gillman SM, Daugherty CA, Gerardo CJ, Lavonas EJ. Validation of Patient Specific Functional Scale in Copperhead Snakebite Patients. Clin Toxicol. 2015;53(7):649.
- 21. Gerardo CJ, Quackenbush E, Lewis B, et al. The efficacy of Crotalidae polyvalent immune Fab (Ovine) antivenom versus placebo plus optional rescue therapy on recovery from copperhead Snakebite envenomation: a randomized, double-blind, placebo-controlled, clinical trial. Ann Emerg Med. 2017;70(2):233–44 e3. pmid:28601268
- 22.
Devellis RF. Scale Development: Theory and Applications. 4 ed. London: SAGE Publications; 2016.
- 23.
Nunnally J, Bernstein I. Psychometric Theory. 3 ed. New York: McGraw-Hill; 1994. 752 p.
- 24. Bland JM, Altman DG. Agreed statistics: measurement method comparison. Anesthesiology. 2012;116(1):182–5. pmid:22129533
- 25. Rankin G, Stokes M. Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clin Rehabil. 1998;12(3):187–99. pmid:9688034
- 26. Maglinte GA, Hays RD, Kaplan RM. US general population norms for telephone administration of the SF-36v2. J Clin Epidemiol. 2012;65(5):497–502. pmid:22269331
- 27. Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health. 2009;12(4):419–29. pmid:19900250
- 28. Chatman AB, Hyams SP, Neel JM, et al. The patient-specific functional scale: measurement properties in patients with knee dysfunction. Phys Ther. 1997;77(8):820–9. pmid:9256870
- 29. Westaway MD, Stratford PW, Binkley JM. The patient-specific functional scale: validation of its use in persons with neck dysfunction. J Orthop Sports Phys Ther. 1998;27(5):331–8. pmid:9580892
- 30. Hefford C, Abbott JH, Arnold R, Baxter GD. The patient-specific functional scale: validity, reliability, and responsiveness in patients with upper extremity musculoskeletal problems. J Orthop Sports Phys Ther. 2012;42(2):56–65. pmid:22333510
- 31. O’Shea SD, Taylor NF, Paratz JD. Measuring changes in activity limitation and participation restriction in people with COPD. Int J Ther Rehab. 2005;12(6):264–8.
- 32. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008;11(2):322–33. pmid:18380645
- 33. Kay M, Santos J, Takane M. mHealth: New horizons for health through mobile technologies. World Health Organization. 2011 Jun;64(7):66–71.
- 34.
Kelly T, Minges M. Information and Communications for Development 2012: Maximizing Mobile, Executive Summary. Washington: International Bank for Reconstruction and Development/The World Bank. Retrieved from http://www.worldbank.org/en/topic/ict/publication/ic4d-2012. 2012.
- 35.
Groupe Speciale Mobile Association. The mobile economy 2014. London: GSMA. 2014.
- 36. Gerardo CJ, Vissoci JR, Brown MW, Bush SP. Coagulation parameters in copperhead compared to other Crotalinae envenomation: secondary analysis of the F(ab')2 versus Fab antivenom trial. Clin Toxicol (Phila). 2017;55(2):109–14.
- 37. Gerardo CJ, Vissoci JRN, Evans CS, Simel DL, Lavonas EJ. Does this patient have a severe Snakebite envenomation?: the rational clinical examination systematic review. JAMA Surg. 2019. [Epub ahead of print] pmid:30758508