Introduction

The International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) exam is recognized universally as the gold standard for the assessment and classification of neurological impairment after spinal cord injury (SCI) [1]. It is the only biomarker considered as a core common data element used in acute clinical SCI trials regardless of intervention, with its applicability to the diagnosis, prognosis, progression, and recovery of SCI [2]. The ISNCSCI exam should be performed within 72 h of inpatient admission and discharge, then at subsequent follow-up visits throughout the lifetime of an individual with SCI [3]. However, reliable and accurate performance of the ISNCSCI requires standardized examiner training and practice, as well as an extensive in-person assessment of the unclothed patient in supine position for 30–60 min. Such requirements can hinder access to assessment in areas far from major centers classified as SCI hubs. Many major SCI longitudinal databases and registries, such as the SCI Model Systems (SCIMS) Database and the Rick Hansen SCI Registry, include ISNCSCI neurological data. However, the percentage of missing ISNCSCI data is notably higher at follow-up than at initial enrollment due to the difficulties enrollees face returning to appropriate centers for in-person exams [4, 5]. This lost data limits the usefulness and power of registries to answer questions on neurological change over time and there is a growing consensus in the field that there is a critical need for a version of the ISNCSCI allowing remote collection of data.

A full ISNCSCI is composed of motor, sensory, and anorectal components used to determine an individual’s neurological level of SCI, zones of partial preservation, as well as motor and sensory scores, that can together classify individuals on the American Spinal Injury Association (ASIA) Impairment Scale (AIS) [6]. The anorectal portion of the ISNCSCI (I-A-ISNCSCI) is essential for distinguishing complete vs. incomplete SCI and is based on the sacral sparing criteria: (1) S4–5 dermatome light touch (LT) sensation, (2) S4–5 pin prick (PP) sensation, (3) deep anal pressure (DAP), and (4) voluntary anal contraction (VAC). One defined as having a complete (AIS A) injury has no sensory or motor function preserved in the sacral segments [7]. Despite training in sensorimotor evaluation, nonphysicians, and physicians in training may be less likely than experienced physicians to feel comfortable performing the critical I-A-ISNCSCI exam. Not only can it be a relatively invasive portion of the exam, but some people with SCI may have difficulty understanding or cooperating with the involved instructions [8, 9]. Further, an online survey of SCI practitioners, including therapists and physicians from various specialties, found marked inconsistency in the performance of the ISNCSCI anorectal exam [10].

Self-report tools and questionnaires have been studied as potential adjuncts or alternatives to the I-A-ISNCSCI exam. Harvey et al. studied participants with chronic SCI in the outpatient setting and suggested participants were reasonably accurate at self-reporting S4–5 sensory and motor function, but with a high false positive rate for S4–5 motor function in those with motor levels below T10 [11]. Prior studies by Zariffa et al. also suggested anorectal findings may be predicted using S1 findings such that S1 motor testing may substitute for VAC with ~85% accuracy, but that this accuracy drops off markedly with levels of injury below T10 [12]. Findings from Zariffa et al., however, do not resolve the need for an in-person assessment and findings from both Harvey and Zariffa studies are of limited relevance for many individuals with SCI levels below T10.

Liu et al. developed a bowel-routine-based self-report questionnaire for assessing sacral sparing after SCI with the idea that patients may find questions using terms associated with their usual bowel management easier to comprehend. They proposed good validity of their tool for 102 study participants all within 12 months post injury (k = 0.79–0.93), except when testing for motor function as related to VAC in those with increased anal sphincter tone [13]. Hamilton et al. added to this literature by examining the accuracy of a self-report survey compared with sacral exam for 116 patients with chronic SCI in an outpatient setting. They suggested self-reported sacral sparing could predict absence of sensation in complete (AIS A) injury with 100% negative predictive value (NPV) and presence of sensation in motor incomplete (AIS D) injury with 100% positive predictive value (PPV). However, overall NPVs ranged from 92% (VAC) to 100% (LT), while PPVs varied even more from 48% (VAC) to 73% (DAP). Hamilton et al. concluded that the self-report of sacral motor and sensory sparing was not predictive enough to rely on for accurate classification [14].

Based on our review of the above literature, our study sought to design an interview version of the anorectal portion of the ISNCSCI exam (I-A-ISNCSCI) and evaluate its use in the acute inpatient rehabilitation (AIR) setting. The AIR setting provides a readily available sample of individuals with SCI who all undergo the ISNCSCI as routine standard of care and, if they agree to participation, could be easily approached by the study team to perform the interview based assessments. The agreement between standard anorectal ISNCSCI exam results (S-A-ISNCSCI) and interview version answers (I-A-ISNCSCI) was tested. If observed agreements are adequate, we could evaluate the use of the I-A-ISNCSCI in our chronic patient population to see how these results compare with prior Liu and Hamilton studies involving more chronic injury. Expanding the availability of adjunct interview based or self-report tools could have a wide impact on the future of assessing and monitoring outcomes after SCI, from expedited screens for clinical trials to remote follow-up of neurological changes.

Methods

Study design and setting

A prospective, single-blinded study design was employed. Institutional Review Board approval (IRB #18-00143) was obtained prior to conducting any study related procedures and informed consent was obtained from each participant. All study activities took place in a large tertiary care hospital in the AIR setting. Portions of this study were presented at the 2019 American Spinal Injury Association Annual Scientific Meeting [15].

Participants

Forty-five eligible patients admitted to AIR for SCI were consented and enrolled to participate in the study from March to September of 2018. Qualifying participants were screened and referred by the treating AIR physician (TB, VH, ME) and were excluded if they were <18 years of age and if they had any cognitive, speech, or language deficits limiting their ability to understand and respond to a verbal questionnaire. The study was restricted to English-speaking participants.

Procedure

A brief 6-item interview version of the anorectal ISNCSCI exam (I-A-ISNCSCI) was developed using branching logic within institutional Research Electronic Data Capture software [16]. Enrolled participants were administered the I-A-ISNCSCI by research staff within a 3-day window of undergoing a standard ISNCSCI (S-ISNCSCI) including its anorectal portion (S-A-ISNCSCI) as part of routine AIR care. Physicians performing the S-A-ISNCSCI were blinded to I-A-ISNCSCI responses documented by research staff and research staff delivering the I-A-ISNCSCI were also blinded to S-A-ISNCSCI results. The S-A-ISNCSCI exams were done as per universal ISNCSCI guidelines: anorectal sensation to LT and PP were recorded as 0 (absent), 1 (impaired, whether hyposensitive or hypersensitive), or 2 (intact) for both left and right sides of the body. DAP and VAC values were recorded as 0 (absent) or 1 (intact) without distinguishing left from right. The physicians (TB, VH, ME) performing S-A-ISNCSCI exams all had had extensive experience performing the ISNCSCI exam and had completed all of the ASIA e-Learning Center InSTeP (International Standards Training e-Learning Program) Modules [17].

Using the I-A-ISNCSCI, research staff asked enrolled participants about their presumed sacral function in terms of sensation to LT and PP at the anomucocutaneous junction as well as the ability to perceive DAP and perform VAC. In the I-A-ISNCSCI, participants were not asked to distinguish left from right. To compare I-A-ISNCSCI responses to S-A-ISNCSCI results, S-A-ISNCSCI values for LT and PP were consolidated into a single score by taking the highest value recorded between the left and right sides (e.g., impaired LT on the left and absent LT on the right recorded as a “1” overall). Items recorded as “I don’t know” were noted but excluded from the agreement analyses because the interview response could mean they were unsure with respect to exam sensation or unsure how to respond despite having some or intact sensation. I-A-ISNCSCI administrators collected any comments from participants in a general notes section whenever a response of “I don’t know” was given (Fig. 1).

Fig. 1: Interview version of the anorectal portion of the ISNCSCI (I-A-ISNCSCI).
figure 1

If participants responded (Yes) to question 1 then (Yes) to question 2 on the I-A-ISNCSCI, they were scored equivalent to intact (2) for sensation to light touch on the standard S-A-ISNCSCI. If participants responded (Yes) to question 1 but (No) to question 2 on the I-A-ISNCSCI, they were scored equivalent to impaired (1) on for sensation to light touch. The same logic was used for sensation to pin prick with questions 3 and 4. With questions 5 and 6 for deep anal pressure and voluntary anal contraction, a response of (Yes) was scored as intact (1), and (No) as absent (0). If participants responded (No) to question 1, the I-A-ISNCSCI administrator would mark question 2 as (No) and skip to question 3. Likewise, if participants responded (No) to question 3, the I-A-ISNCSCI administrator would skip to question 5.

Full size image

Statistical analyses

Statistical analyses were performed using SPSS version 22.0 and R version 3.5.1. Agreement between S-A-ISNCSCI results and I-A-ISNCSCI responses was evaluated using Cohen’s Kappa (k). The sample was bimodal for age therefore age was reported as an overall mode (with min–max values). Duration of injury was reported as a median (also with min–max values). Given a small sample size, all other data were summarized using counts (N) and percentages (%). Cohen’s Kappa (k) was interpreted according to the following criteria: 0.21–0.40 was deemed fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1.00 almost perfect [18].

Results

The S-A-ISNCSCI was administered to all 45 enrolled participants as part of standard AIR care, whereas the I-A-ISNCSCI was administered to 40 of these participants. Five participants were excluded from analyses because they missed the 3-day window from their S-A-ISNCSCI to be administered the I-A-ISNCSCI. The study sample was bimodal for age with peaks in the 30–35- and 50–55-year ranges. Further participant demographics and injury information are in Table 1.

Table 1 Study participant demographics and injury information (total N = 40).
Full size table

Items recorded as “I don’t know” were noted but excluded from agreement analyses. Agreement between S-A-ISNCSCI results and I-A-ISNCSCI responses was substantial for S4–S5 sensation to LT (k = 0.71, 95% CI 0.52–0.90, N = 36), PP (k = 0.68, 95% CI 0.48–0.87, N = 38), and DAP (k = 0.77, 95% CI 0.53–1.00, N = 37); but only fair for VAC with insufficient statistical significance (k = 0.29, 95% CI −0.01 to 0.59, N = 36) (Table 2A). Agreement was also substantial for the identification of overall injury completeness vs. incompleteness based on all ISNCSCI sacral sensory criteria combined (k = 0.72, 95% CI 0.47–0.97, N = 40) (Table 2B). Despite efforts to deliver the I-A-ISNCSCI to participants equally within the 3-day window from their S-A-ISNCSCI, the majority (31) of the 40 participants who responded to the I-A-ISNCSCI did so after undergoing the S-A-ISNCSCI on admission as part of routine AIR workflow.

Table 2 Agreement between S-A-ISNCSCI results and I-A-ISNCSCI responses (N = 40).
Full size table

Discussion

The ISNCSCI exam is the current gold standard for determining the level and completeness of SCI but an adjunct interview based version of it allowing expedited and remote collection of data could be of great value within the field. We sought to add to the existing literature to date by designing our own interview version of the anorectal ISNCSCI exam in the AIR setting.

Our pilot data suggests that in the AIR setting, agreement between S-A-ISNCSCI results and I-A-ISNCSCI responses was substantial in terms of S4–S5 sensation to LT and PP, the perception of DAP on digital rectal exam, and the overall identification of completeness vs. incompleteness of injury based on combined ISNCSCI sacral sensory criteria. Comments regarding issues with the VAC item collected from participants included: “patient was confused about time frame of question, he doesn’t know if he should answer the question based on his current/past status”, “patient doesn’t know what abnormal and normal means”, and “patient didn’t like the DAP/VAC questions, was more comfortable with and responded better to situational types of questions tried after”. The majority of participants who responded to the I-A-ISNCSCI did so after undergoing the S-A-ISNCSCI on admission as part of routine AIR workflow.

Individuals with SCI in AIR undergo the ISNCSCI exam often more than once on admission (e.g., by a resident physician on arrival to the unit then by an attending physician to confirm findings during subsequent rounds), again on discharge, then however many more times as needed if any change in neurological status is suspected. Repeated ISNCSCIs are not thought to be invalidated by prior exams in clinical practice. Longitudinal follow-up assessments such as those performed for SCIMS are other cases, in which individuals undergo serial ISNCSCI exams that are not considered to invalidate each other. We wanted the I-A-ISNCSCI to be administered within a 3-day window from the S-A-ISNCSCI for consistency in clinical condition, but did not strictly enforce whether it should be before or after the S-A-ISNCSCI. From an assessment tool development perspective, the risk of recall bias could have been better evaluated in our study. The majority of participants received the I-A-ISNCSCI after the S-A-ISNCSCI often because the research team was unable to approach participants with the I-A-ISNCSCI before clinicians performed the S-A-ISNCSCI as part of routine admission exams. Patients are not usually presented the scores of their ISNCSCI sacral sparing assessments in AIR, but they may still recall in general what sensations they experienced during the exam leading to some recall bias. Had we a more even distribution of when the I-A-ISNCSCI was administered within our 3-day window, we could have better determined how big of a role recall bias played in our study.

Another point worth discussing is why we felt items initially recorded as “I don’t know” should be excluded from the agreement analyses. In clinical practice, if an individual cannot distinguish PP from LT, PP is scored as a 0 (absent) and if a patient demonstrates inconsistency for a given item, this too is scored as 0 (absent) [6]. One might therefore think counting answers of “I don’t know” as “No” is consistent with how the ISNCSCI is normally done. However, this response could mean participants were unsure with respect to exam sensation or unsure how to respond despite having some or intact sensation. Having an “I don’t know” option on the I-A-ISNCSCI helped us to find items that needed further clarification or revision but would not be appropriate for including in the agreement analyses.

We designed our study hoping the AIR unit could be a convenient and effective setting for developing and evaluating the validity of a tool such as the I-A-ISNCSCI. AIR was convenient for delivering the tool to participants, but had logistical limitations such as difficulty administering the I-A-ISNCSCI prior to S-A-ISNCSCI exams making it difficult to rule out recall bias as a confounding factor. AIR might be an appropriate setting for designing other adjunct tools like the I-A-ISNCSCI such as interview based versions of the motor and sensory portions of the ISNCSCI (e.g., an I-M-ISNCSCI and an I-S-ISNCSCI) with direct patient feedback. Individuals in AIR could be consulted regarding the wording of questions and used to test trial versions of the tools. The chronic outpatient setting may, however, be more appropriate for testing validity as it could help control the timing of tool administration and test the tool’s performance on participants who are more experienced with their SCI and neurological function.

Our study had the above limitations and only evaluated individuals in the AIR setting therefore its findings are not generalizable to the vast population of individuals living with chronic SCI. In subsequent attempts of our study, we may use AIR to develop further tools like an I-M-ISNCSCI or I-S-ISNCSCI but will need to better account for the aforementioned limitations to truly assess the validity of such tools. This includes extending the study to participants in the chronic setting to determine the potential for using the I-A-ISNCSCI and similar tools for longitudinal clinical or research follow-up. Eligible participants recruited via SCI newsletter advertisements or provider recommendations could be enrolled either by phone or e-mail before their scheduled outpatient visits and be administered a tool like the I-A-ISNCSCI within 3 days before or just prior to their visit during which an S-A-ISNCSCI is done as part of routine follow-up care.

There is a significant need in the field to continue exploring ways to facilitate the standardized assessment of individuals with SCI. The current literature remains sparse with regards to how we can best study interview based and self-report approaches to the ISNCSCI exam that can serve as effective adjunct clinical tools. Expanding the research into and availability of interview based or self-report tools can have wide effects on the future of assessing and monitoring outcomes after SCI, from expedited screens for clinical trials to remote long term follow-up of neurological changes (also comorbidities like carpal tunnel syndrome or syrinx development). The use of interview based or self-report supplements for assessing individuals with SCI remains worthy of further investigation and we hope that experiences from our pilot study can help serve as a methodological guide to those within the community seeking to carry out similar studies and add to the existing literature.

Conclusion

Our pilot data suggests substantial agreement between the S-A-ISNCSCI and I-A-ISNCSCI in the evaluation of sacral sensation to LT, PP, DAP, and completeness of injury based on combined ISNCSCI sacral sensory criteria. This pilot study was a first step in the development of interview based tools such as the I-A-ISNCSCI in an AIR setting providing convenient access to individuals with SCI and their direct feedback. However, the study design introduces potential recall bias and may not match true clinical situations such as remote follow-up of neurological changes for chronic patients. The use of interview based tools for assessing individuals with SCI remains worthy of further study.