Introduction

Shoulder pain is common and often poorly treated [1]. Patient-reported outcome measures (PROMs) [2] are used to assess symptomatic and functional change in shoulder disorders. For example, the shoulder pain and disability index (SPADI) is commonly used in clinical trials, comprising five pain and eight disability questions with severity recorded on ordinal scales [3]. It is reliable, responsive and has good construct validity. While analysis of ordinal PROM scores may provide discriminatory ability between treatments at group level, this may not reflect therapeutic success for the individual. The patient acceptable symptom state (PASS) cut-off is the value on a PROM beyond which the patient feels good enough to continue in that state [4, 5]. Establishing PROM cut-offs for PASS would indicate therapeutic success at the individual level and provide a tool for standardising responder rates in clinical trials.

There are limited data for PASS values for shoulder outcomes [6] with existing studies assessing the cross-sectional relationship between PASS and PROMs, including the American shoulder and elbow surgeons (ASES) score, the simple shoulder test (SST), and the Visual Analog Scale (VAS) pain score [6, 7], with only one assessing SPADI in a shoulder pain cohort awaiting surgery [8].

There are no prospective longitudinal studies investigating SPADI PASS cut-offs. We aimed to determine PASS cut-offs using two different methods and assess whether these values were influenced by time or other covariates.

Materials and methods

Study population

Patients care with shoulder symptoms are initially managed in the community, but if they continue to have problems, they are referred for a diagnostic ultrasound scan. Patients attending a hospital, outpatients’ clinic, for their first shoulder scan were recruited from 2016 to 2017. Inclusion criteria were: age > 18 years, ability to provide informed consent, first ever ultrasound scan of shoulder (determined by clinical records). Exclusion criteria were: inflammatory arthritis; previous fracture/dislocation/surgery of the affected shoulder; steroid injection or physiotherapy for target shoulder within prior 6 weeks (to exclude effects of treatment, which are short term [9], on baseline symptoms); or complex regional pain syndrome. To avoid recruitment bias, quota sampling was used in four categories: gender (male/female) and age (younger/older split at the median); target proportions of these categories were determined from a previous study [10]: males aged < 54 (25%), males aged ≥ 54 (22%), females aged < 54 (25%), females aged ≥ 54 (28%).

Data collection

Patients completed standardised questionnaires at baseline and 6 months, assessing age (years), gender, body mass index (BMI), duration of symptoms, pain status (yes/no), other painful body joints, hospital anxiety and depression scale (HADS) [11] and SPADI. Treatments received (physiotherapy, corticosteroid injections and/or surgery) prior to initial scan were recorded at baseline and treatments received during the study were collected at 6 months. Ultrasound findings were collected to understand the aetiology of shoulder pain.

To determine PASS, patients were asked “in the next few months, if you were to remain as you were during the last 48 h, would this be acceptable or unacceptable to you?” This is the wording recommended by an Outcome Measures in Rheumatology (OMERACT) special interest group [12] and dichotomises each participant into PASS positive (PASSpos) or negative (PASSneg).

Statistical analysis

SPADI scores were transformed to interval scaling by assessing fit to the Rasch measurement model in RUMM 2030 software [13, 14].This process formally tests that the data meet minimum requirements for measurement then allows transformation from ordinal to interval linear scaling [15, 16], facilitating parametric analysis [17]. Comparisons were made between included and excluded patients using Student’s t, Mann–Whitney U or Chi-square tests according to data type and distribution; visual inspection of approximate fit to the normal distribution was performed for continuous variables. PASS cut-offs were calculated using the 75th percentile of the cumulative percentage curve of patients who were PASSpos [5] and the point on the receiver operating characteristic (ROC) curve, which maximised the Youden index [18, 19]. The diagnostic accuracy of each method was reported. We simulated some data to investigate potential differences between the 75th percentile and ROC methods (see Online Resources 3 for more details).

The association between PASSpos/PASSneg and SPADI was assessed using binary logistic regression. An interaction was added between baseline and 6 month SPADI to evaluate if 6 month PASS cut-off depended on baseline score. Interactions with other baseline covariates were also explored; main effects and interactions with 6 month SPADI scores were added for age, gender, baseline painful joint count and baseline HADS scores. If an interaction was not statistically significant, both the interaction and the main effect were removed. To illustrate the influence of baseline score on PASS at 6 months, cut-offs were calculated separately within tertiles of baseline SPADI. SPADI reliability was evaluated using standard error of measurement (SEM) and smallest detectable change (SDC) calculated using Cronbach’s alpha.

Results

SPADI showed acceptable fit to the Rasch model (Online Resource 1). To create a unidimensional total score combining the pain and disability subscales, 7% of the unique variance was lost in a bi-factor equivalent solution. There was no evidence of substantive differential item functioning by age, sex, shoulder side or visit, the latter indicating that the measurement scale remained invariant over time.

Table 1 shows baseline characteristics of patients included in or excluded from the analysis. Questionnaires were returned by 336/500 participants (67%) at 6 months; 304/500 participants (60%) had full data at both time points. Patients excluded from longitudinal analysis were younger with higher HADS, but other covariates were similar. Bursitis was the most common finding on ultrasound at baseline (71%), followed by impingement (63%), acromio-clavicular joint degeneration (53%) and rotator cuff tendinopathy (34%). Amongst the patients included in the analysis, at baseline 69/304 (23%) were PASSpos. At 6 months, 193/304 (63%) were PASSpos; mean ± SD 6 month SPADI scores were total 39.6 ± 19.1, pain 36.5 ± 20.0, disability 33.6 ± 20.3.

Table 1 Baseline characteristics of those included in or excluded from longitudinal analysis (total n = 500)
Full size table

Reliability results for SPADI scores are presented in Table S1, Online Resource 2. Most of those reporting improvement > SDC were PASSpos at 6 months: SPADI total 95/106 (90%), pain 100/119 (84%) and disability 113/130 (87%). At baseline the PASS cut-offs for total SPADI were 49.8 and 46.9 using the 75th percentile and ROC method, respectively (Table 2). At 6 months, PASS cut-offs did not differ from baseline to an extent that exceeded measurement error.

Table 2 Unadjusted PASS cut-offs for SPADI (n = 304)
Full size table

The association between 6 month SPADI and PASS did not vary by age, gender, painful joint count or HADS scores (all interactions p > 0.05, data not shown), but at higher baseline SPADI, the coefficient for the association between 6 month SPADI and PASS was larger (p = 0.036; Fig. 1a). This was driven by disability (Fig. 1b), rather than pain (Fig. 1c). Patients with more severe disability at baseline were willing to accept a comparatively greater degree of disability at 6 months. In contrast, there was an upper limit of pain that was acceptable at 6 months, irrespective of pain at baseline.

Fig. 1
figure 1

Contour plots showing probability of PASSpos at given values of SPADI at baseline and 6 months. a Shows the influence of SPADI total at baseline on the association between 6 month SPADI total and reporting PASSpos. b Shows the influence of SPADI pain at baseline on the association between 6 month SPADI pain and reporting PASSpos. c Shows the influence of SPADI disability at baseline on the association between 6 month SPADI disability and reporting PASSpos. In all figures, the lowest probability (0.0–0.2) is highlighted in blue, probability between 0.2 and 0.4 highlighted in cyan, probability between 0.4 and 0.6 highlighted in green, probability between 0.6 and 0.8 highlighted in yellow and the highest probability (0.8–1.0) shown in red. In these plots, completely horizontal shaded bands would indicate no influence of baseline SPADI on the association between 6 month SPADI and PASS. b Shows very little effect of baseline SPADI pain; the disability subscale (c) drives most of the effect seen for total SPADI

Full size image

When baseline SPADI scales were divided into tertiles, the more symptomatic a patient was at baseline (higher score), the higher their SPADI PASS cut-off was at 6 months (Fig. 2). Differences in pain cut-off did not exceed SDD; however, the effect was more pronounced (> SDD) for disability. On the native (ordinal) scale, differences in cut-off were greater than on the Rasch-transformed interval scale, emphasising the importance of using interval scaling (Table S2, Figure S1, Online Resource 2).

Fig. 2
figure 2

Variation in 6 month SPADI PASS cut-offs according to baseline (BL) SPADI, by each of two methods, using a interval scaling, b ordinal scaling. A shows the variation in SPADI cut-off for PASS at 6 months using the 75th percentile and ROC methods for SPADI total, pain and disability scores for each tertile [low (light grey), moderate (dark grey) and high (black)] of baseline SPADI: using the Rasch-transformed SPADI scores. b Shows the variation in SPADI cut-off for PASS at 6 months using the 75th percentile and ROC methods for SPADI total, pain and disability scores for each tertile of baseline SPADI as described above using the native untransformed SPADI scores

Full size image

Discussion

This is the first study to evaluate the PASS threshold for SPADI longitudinally in a community-based cohort. We found that PASS cut-offs varied between the ROC and 75th percentile methods. PASS cut-offs did not change substantively over time. However, the PASS cut-off on the total SPADI score at 6 months was dependent on the baseline score; this was primarily driven by the disability subscale, rather than the pain subscale.

Our SPADI PASS cut-offs differed from the only other study assessing PASS and SPADI, although that cross-sectional cohort recruited post-surgical patients [10]. Our cut-offs also varied according to the methods used; the optimum methods for calculating PASS are unknown and may depend on the purpose [18]. The 75th percentile method may be inappropriate for determining PASS changes longitudinally, because the scores within those reporting PASS may decrease, affecting the location of the 75th percentile. PASS provides useful information on therapeutic success at the individual level, but does not reflect the desire to improve: patients may define their state as acceptable, but want to feel better. Given the negative and positive predictive values from our study (Table 2), the SPADI PASS cut-off may be more useful as a descriptive outcome for trials rather than a treatment target.

Age and gender did not modify the association between 6 month SPADI and 6 month PASS after adjusting for baseline score; however, these tests may have lacked power as testing for these interactions would require higher number of participants. Previous studies using different PROMs found age and gender affected PASS cut-off, but did not test for interactions or adjust for other covariates or baseline score [20, 21].

Disease adaptation may alter interpretation of a questionnaire, thus changing a patient’s individual PASS threshold over time. This response shift can only be assessed if the measurement properties of the questionnaire itself are time invariant, which we confirmed for SPADI. Patients with worse symptoms at baseline reported higher PASS cut-offs at 6 months. Existing research, using different scales, found that those with worse initial activity limitation in acute rotator cuff syndrome were shown to have higher PASS cut-offs at follow-up, possibly due to adaptation, whilst their pain cut-offs did not differ by baseline score, consistent with our findings [22]. Another study reported data consistent with this trend in those undergoing hip replacement [23]. Studies have found PASS cut-offs for other scales to be stable over time [20, 24], whereas another found thresholds declined [21]. PASS cut-off was reported to be stable after 6 months from knee replacement [23]. This variation in findings may be due to treatment expectation or PASS thresholds may be disease or treatment specific. Another source of variation might be the use of the 75th percentile method to identify changes in PASS cut-off over time, which, as we have shown in Online Resource 3, can give variable results depending on the level of change in symptoms, even if there is actually no change in ‘true’ PASS cut-off.

We did not adjust our analysis for treatments received between baseline and follow-up, because we needed to account for the total effect of baseline SPADI on the association between SPADI and PASS at 6 months. The level of baseline symptoms would potentially influence treatments received post-baseline; therefore, including such treatments in the model may distort this association.

This paper has several strengths. While previous studies used different wordings and time anchors for PASS [7, 10] we used the timeframe of “next few months” recommended by OMERACT [12]. Standardisation of the wording is important for comparison across studies and variations may influence results [18]. SPADI underwent Rasch transformation enabling inferential testing by converting ordinal scores to interval scaling and our measures showed very good reliability. Other studies did not transform their data; therefore, they may have underestimated or overestimated the extent of cut-off dependence on initial values due to the nonlinearity of estimates obtained from these PROMs.

There are limitations to this study. The 6 months response rate was 67%. However, this is better than a published overall rate for 1607 surveys of 48.3% [25]. Our non-responders were on average younger with more anxiety and depression, consistent with published predictors of response to surveys [26]. However, the baseline SPADI and PASS were similar between responders and non-responders. Pain duration, BMI, injections received and sex were also similar between the two groups. This study involved self-completion of questionnaires, which may introduce recall bias.

This study has shown for the first time that the PASS cut-off for SPADI varies depending on calculation method and symptom severity at baseline. Understanding the PASS threshold is important for clinical research as the relationship between SPADI and objective disease parameters is ill-defined. Further work in different centres is required to substantiate these findings.

Availability of data and materials

The data that support the findings of this study are available from the corresponding author upon reasonable request.