Testimony at court: a randomised controlled trial investigating the art and science of persuading witnesses and victims to attend trial

Abstract

The presence of civilian witnesses and victims in court is central to the effective operation of the criminal justice system. However, there is evidence of significant non-attendance which can result in ineffective and cracked trials. To address this, West Midlands Police Witness Care Unit and the Behavioural Insights Team designed an intervention using behavioural insight principles consisting of (1) a new conversation guide for Witness Care Officers (WCOs); (2) a redesigned ‘Warning Letter’ confirming details of the proceedings; and (3) a new reminder call and SMS. The impact of the new approach was evaluated through a randomised controlled trial in which 36 WCOs were randomly assigned to either “business as usual” (control) or treatment. The evaluation used an intention-to-treat design with implementation guided and encouraged at several points. Subgroup analysis was undertaken to explore whether differential effects were seen for domestic violence cases or between those that were victims and witnesses. Results indicated that the treatment approach was directionally positive in all cases, but that the increase in attendance was not statistically significant. This is in line with findings of other similar research in this area.

Introduction

The attendance of victims and witnesses in court is a critical factor in the effective operation of the criminal justice system. The number of cases that are dealt with by the court system in England and Wales is substantial, with caseload figures fluctuating between 350,000 and 400,000 per quarter for the Magistrates’ court and 30,000 and 40,000 per quarter for the Crown court. Whilst receipts of caseloads are declining as a general trend, the number of court disposals are inconsistent, meaning that outstanding cases at the end of each quarter show considerable variation. Taking cases to court is a significant public cost and the cost of crime related to the UK criminal justice system (including policing) was approximated at £11.6 billion annually in 2000 (Brand and Price 2000). There is also evidence that suggests that for nearly all major Home Office offence categories, the average public cost associated with the courts, prosecution service and prisons out-weighs the policing resource expenditure (Brand and Price 2000).

Some national information on the number of trials that were impeded by, or that did not proceed as a result of, victim or witness non-attendance is provided by the Ministry of Justice (MoJ). Recent figures counting the number of trials that were effective or otherwise indicate that in the last quarter of 2017, 35% of Crown Court trials were cracked¹ and 15% were ineffective,² amounting to roughly half of the caseload (MoJ Criminal court statistics: October to December 2017). Whilst non-attendance of a witness or victim is not the only reason for a cracked trial it is acknowledged that this is one of the main driving factors (HM Crown Prosecution Service Inspectorate Report 2016). Taking a local policing area perspective, West Midlands Police (WMP) data shows that approximately 16% of civilian victims and witnesses do not attend court hearings when they are required to (according to the Witness Care Unit’s (WCU) 2015 baseline figures), which results in ineffective and cracked trials. This has a large impact on victims and other witnesses, and has considerable cost implications for the courts, the Crown Prosecution Service (CPS), Police and other related services. Ultimately it also means that justice is often not served in these cases and the defendant is not held to account for their actions.

There is a dearth of research that specifically explores reasons for failure to attend court. Suggested reasons include fear of the court process, fear of repercussions, reluctance to ‘stitch someone up’, a lack of time or presence of other commitments, a bad previous experience with court or forgetting the time and date or the request altogether (e.g. Belknap and Graham 2000). It could also be the case that certain vulnerable witness/victim groups are less likely to have the resources to enable them to attend court or be confident in their role. Indeed, the cost of attaining transport to court or the lost wages resulting from a court attendance might be prohibitive to those from lower income groups. Hence, court attendance might be lower in certain social and economic groups or areas. For example, a study by Mateyoke-Scrivner et al. (2004) demonstrated that lower levels of education consistently predicted higher dropout from drug courts.

Studies that have been done on witness and victim reluctance to attend court have mainly focused on domestic violence and assault cases. These suggest that in this context, the main reason for reluctance to cooperate with court processes is a fear of reprisal (e.g. McLeod 1983; Belknap and Graham 2000). A further explanation is that witnesses or victims may lack confidence in the criminal justice system in general, or the courts specifically, or believe that the courts lack legitimacy. Research in domestic violence cases suggests that engaging the victim in the justice process is often crucial to conviction- the most important variable in predicting a guilty verdict for the cases reviewed was the number of times the victim met with the prosecutor (Belknap and Graham 2000). Unfortunately, this was in a context where court professionals reported distrust of victim advocates and their decisions were often driven by financial costs. The report concluded that training of court professionals would be an effective intervention to encourage victim engagement.

There is a lack of evidence in terms of tested interventions to encourage witness and victim attendance at court. Arguably, some courts are set up to deal with some of the possible reasons for non-attendance more directly and with more ease than others. Interventions that deal with victim vulnerability and fear or education of professionals could be very effective but will generally require long time scales and sustained investment of resources and considerable financial support. On the other hand, a simple, low cost intervention might be the sending of reminders to minimise the risk that victims or witnesses forget their court date. In this paper we focus on evaluating a more immediate, low cost intervention along these lines. However, we argue that a factual prompt alone is unlikely to change attitudes to, and therefore behaviour as a result of, a court summons. It follows that it is prudent to consider not only whether to send communication, but also the nature of that communication in terms of its potential in altering behaviour.

For many years, researchers in the field of psychology have considered different strategies with the potential to change an individual’s attitude and behaviour towards a presenting stimulus. For example, the principles of behaviourism explore the roles of reinforcement, punishment and reflexes on behavioural responses (e.g., Skinner 1948, 1971). Recently, there has been an increased focus on human decision-making processes and in particular cognitive biases that might occur as a result of the short cuts our unconscious minds take to speed up the process (Kahneman 2011). The implication of this is that behavioural science approaches that give us a ‘nudge’ might be sufficient to change a resulting behavioural outcome. This type of nudge has become a key strategy of the Behavioural Insights Team (BIT) which has seen success in changing behaviour in many social policy areas (Halpern 2016). When examining the effectiveness of nudges in achieving desired outcomes, experimental approaches need to consider: firstly, the appropriate experimental design; and secondly, the particular context in which such a strategy might be implemented.

Previous research has used a behavioural science approach to encourage court attendance of people issued with tickets for low level offences (Cooke et al. 2018).³ This involved the redesign of the court summons letter and the addition of a text message reminder. The intervention aimed to reduce the number of missed court dates in a bid to reduce expensive arrest warrants that were a consequence of non-attendance. The redesigned form reduced failure to attend by 13%, which translated to preventing around 17,000 warrants being issued in the New York City area. The results of this study are at odds with those of Chivers and Barnes (2018). In this trial, a treatment group of 474 defendants in the Hampshire and the Isle of Wight (UK) were sent a text message the day before their court appearance, whilst the control group (n = 472) received the standard postal notification. The statistical analysis found no significant effect of the treatment on either attendance at court or the number of failure-to-appear warrants issued. These two previous trials differed in terms of sample size, evaluation design and outcome. It appears therefore, that there is mixed evidence regarding the effect of nudge approaches on defendant court attendance. The Behavioural Insights Team also ran an RCT sending SMS reminders to defendants who had been granted bail by the police prior to their initial hearing at Magistrates’ Court. The SMS informed them of their hearing date and time and provided them with a map with travel instructions. The impact of the trial will be assessed using a binary variable of whether a Failure to Appear (FTA) arrest warrant is issued for non-attendance, and the results are forthcoming.

To the authors’ knowledge there has to date been only one published paper examining the effectiveness of nudging on victim attendance at court (Cumberbatch and Barnes 2017). This research specifically examined whether a mobile phone text (SMS) message to victims or witnesses 2 to 3 days before their scheduled court appearance increased attendance in court in minor criminal cases. This research randomly assigned the text message treatment to 405 of 811 victims who were sent a postal notice to attend court in Staffordshire, UK. An important aspect of the message was that it was short, personalised and contained the key practical information about the court hearing. The statistical analysis demonstrated that there was no significant difference in the non-attendance rate between the treatment and control group (24% and 22% respectively). The authors concluded that the text message was not a sufficient nudge to encourage court appearance.

This paper reports the findings of a nudge-based experiment to encourage victim and witness attendance at court implemented by West Midlands Police. It differs from the Cumberbatch and Barnes research in two notable ways. First, the intervention was implemented by Witness Care Officers (WCOs), who were responsible for communicating with victims and witnesses. WCOs randomly selected to take part in the trial were trained in applying all aspects of the treatment. In addition, the intervention tested in this trial was more comprehensive and involved a three-pronged approach: (1) a new conversation guide that WCOs could refer to when they communicate with witnesses and victims; (2) a redesigned warning letter; and (3) a new reminder call and SMS message. The actions of the WCO have a large part to play in victim and witness engagement and this trial tests whether a modified procedure improves outcomes in court (in terms of witness/victim attendance).

Background and approach to intervention and evaluation

When planning evaluation research, thought needs to be given to the most appropriate type of experimental design. In many cases, particularly when a reasonably large pool of individuals is available for sampling and resources are available to implement appropriate systems of allocation, randomisation is the most likely method to ensure a high quality evaluation that minimises threats of certain biases and threats to internal validity (e.g. Shadish et al. 2002).

Given the particular context-described in detail below-it was decided that a randomisation procedure would be appropriate in this study. The randomised controlled trial was designed to test the impact of an intervention aimed at improving victim and witness court attendance by those assigned to the West Midlands Police’s Witness Care Unit in England, UK. The role of this unit is to warn witnesses and victims that they are required to attend court once they have received notification from the Crown Prosecution Service, and to ensure that they attend. The WCU also provides support to witnesses and victims including: offering them special measures at court; referring them to Victim Support; and providing them with assistance for getting there on the day (such as guidance on transportation and childcare).

The Behavioural Insights Team were commissioned by the Dawes Trust to design and evaluate the interventions. Initial discussions identified a number of issues within the current practices that could be improved to help encourage witness and victim attendance in court. The WMP’s WCU had a high number of cases to manage at any one time. In 2015, the WCU processed 5728 cases where 8322 witnesses and victims were required to attend Magistrates’ Court. In other words, a typical annual caseload for a WCO in 2015 was approximately 150 cases with over 200 witness/victim attendances.

Discussions revealed that due to this high demand, WCOs have a backlog of cases and therefore need to focus on processing cases in short timescales. Most communication was conducted via phone calls, although WCOs frequently were not able to get hold of witnesses and victims this way, which left them to rely on communication via letter, email or SMS. There was also variation amongst processes and procedures undertaken by WCOs. Although a standard operating procedure existed, it was very long and did not reflect actual staff practice. Finally, there were issues with timing. Discussions with the WCU management and individual WCOs revealed that the majority of engagement with witnesses and victims occurred weeks or months before the trials take place. In many instances, warning letters were sent to witnesses and victims a few months in advance of the trial date, with no proactive communication from the WCU in the run-up to the trial itself.

As mentioned in the introduction, an important element of the research was to apply a behavioural insights approach to the intervention tested. After considering the specific context the following behavioural insight principles were employed as part of the intervention:

• Implementation intentions (Duckworth et al. 2013): People are more likely to do something when they specify how, when and where they will do it. The new approach was designed to prompt witnesses and victims to plan for their attendance at court: “How will you be travelling to court on 27 March?”;

• Salience (Dolan et al. 2012): The call guidance and Warning Letter (see Appendices 1 and 2) in the new approach include repeated mention of the trial date, time, and address, to increase its salience to witnesses and victims. The call guidance also prompted WCOs to encourage witnesses and victims to write the details down, which was hoped to increase the likelihood of attendance: “Would you like to take a moment to fetch your calendar to write that time down, and check whether there’s anything that you’ll need to now rearrange on that day?”;

• Reciprocity (Buell and Norton 2011): People have an inherent desire to reciprocate—in other words, to give back when they receive something or ‘return the favour’. It is this instinct that drives the impact of ‘operational transparency’ (showing someone the work being carried out on their behalf) on perceived value and satisfaction. By highlighting to witnesses and victims that “lawyers and police have been working hard in recent weeks to ensure that the case is ready to come to court”, it was hoped that the new approach will increase the likelihood that witnesses and victims will reciprocate by attending court;

• Social norms (Schultz et al. 2007): People use other people’s behaviour as a cue for what is acceptable and desirable. The new approach aimed to reduce victims’ and witnesses’ apprehension about attending court by reminding them of the number of other witnesses who manage to give evidence in the West Midlands area: “around 700 people each month give evidence at Magistrates’ Courts in the West Midlands area”; and

• Timeliness (e.g. Judah et al. 2009): When it comes to communication, timing matters: the same message delivered at different times can have drastically different levels of success. The new approach aimed to be more timely by reminding people about their upcoming court attendance at a moment (1 week beforehand) that gave them enough time to make necessary plans (such as arranging travel, a work absence, or childcare) to travel to court on the day but was also near enough that the date was salient in the immediate future.

These principles are useful at understanding the mechanisms by which it was considered that a change in behaviour would occur and demonstrates that there was a theoretical basis for the approach taken (for more information see Behavioural Insights Team 2014).

Experimental design

This section covers the details of the intervention and the experimental approach. It first describes the materials produced using the behavioural insight principles. It then discusses the approach taken to sample selection, identifying participant eligibility and the randomisation process used. Finally, it gives some details of the intended outcomes and how these were measured, along with a description of implementation and intervention tracking.

Materials

Applying the behavioural insight and nudge principles mentioned in the previous section resulted in a new approach consisting of the following three elements:

• 1. A new conversation guide for WCOs to use when first ‘warning’ a witness or victim that they need to attend court (after the defendant has pleaded not guilty at the first hearing at the Magistrates’ Court and the Prosecutor has decided to call the witness) (see Appendix 1);

• 2. A redesigned ‘Warning Letter’ for witnesses and victims confirming the details of the court proceedings and helping them to plan their attendance at court (see Appendix 2); and

• 3. A new reminder call and SMS (with associated conversation guide and template) that will remind witnesses and victims that they need to attend court approximately 1 week ahead of the trial (see Appendices 3 and 4).

WCOs following this new approach were in the ‘treatment’ group. Those in the ‘control’ group continued with their normal long-standing approach (in which they used the original Warning Letter (see Appendix 5) and did not follow set conversation guides or consistent reminder processes).

Sample selection and randomisation

The sample frame for the trial was all adult civilian witnesses and victims who were allocated to the West Midlands Police Witness Care Unit and were sent court warnings during the treatment period, which was between 10 February and 15 July 2016. The sample included only those who were required to attend Magistrates’ Court to give evidence, as some witnesses and victims assigned to a case are not required to be present. Individuals on cases with a guilty plea or cases that were discontinued or withdrawn were not included in the trial. Police and expert witnesses were also excluded from the sample. The sample covered all cases held at Magistrates’ Court in the West Midlands area of the UK (this included trials in Birmingham, Dudley, Sandwell, Walsall, Wolverhampton and Coventry).

As already mentioned, it was considered that randomisation of treatment would be appropriate in this trial. After consideration of the context of the intervention, it was decided that the experiment would be undertaken as a cluster-randomised trial, with randomisation at the WCO level. The rationale for randomising at the WCO level is that the treatment involves the interaction between the WCO and the victim or witness. WCOs who were selected were trained to apply a ‘holistic’ approach to encouraging attendance. Therefore, the delivery of treatment was the service applied by the WCO and not, for example, an individual text message as was the case with the Cumberbatch and Barnes (2017) study. Once they were trained, WCOs could not ‘forget’ the training and hence all victims and witnesses whose caseloads were assigned to a selected WCOs were identified as treatment individuals.

Each WCO was assigned a number from 1 to 36 in order to be randomly allocated to the treatment or control group. The randomisation was conducted to ensure balance in the groups. WCOs who were job share partners and trainers/trainees were allocated to the same group to minimise spill-overs and logistical complications on the job.

Outcome measurement

The aim of this trial was to increase the attendance rates of civilian witnesses and victims required to attend Magistrates’ Court, after being exposed to the treatment. The primary outcome measure was therefore identified as the percentage of witnesses/victims that successfully attend at least one hearing at the Magistrates’ Court at which they were required to give evidence. The percentage of witnesses/victims that successfully attend all hearings at the Magistrates’ Court at which they are required to give evidence was included as a secondary outcome measure. This was because it was believed that attendance is less likely in situations where witnesses and victims were required at multiple hearings, due to feelings of frustration with trial adjournments.

In crime policy research it is often the case that interventions can have crime-specific effects. For example, it is well known that CCTV is more effective at reducing vehicle crime than other types of property crime (e.g. Welsh and Farrington 2009). Subgroup or moderator analysis was therefore planned to identify whether the intervention was more or less effective when considering cases that involved domestic violence. As previously highlighted in the introduction, domestic violence witnesses and victims are a particularly vulnerable group in court settings. In fact, their likelihood to fail to participate in prosecutions has in some contexts led to legal mandates to give evidence which in turn have led to questions concerning human rights (e.g. Hanna 1996). Data from the WCU, both before and during our trial, supports the claim that attendance of witnesses and victims is significantly lower for domestic violence cases. Given the particular challenges concerning court attendance of domestic violence witness and victims, it was deemed appropriate to investigate the particular effect of the intervention on this sub-group.

A final set of subgroup analyses focus on the distinction between the two populations covered by the WCOs. For example, it could be the case that victims responded differently to the treatment than other witnesses (i.e. witnesses that are not victims). These different groups have different stakes in the trial and different relationships with those involved in the court cases (e.g. Bottoms and Roberts 2010 includes interesting discussions about the particular challenges of being a victim in a court of law). Given these different circumstances, it could be that the nudge-based behavioural approaches will work more effectively for one group than for the other.

Implementation and intention to treat

Before the implementation of the trial, BIT evaluators trained WCOs in small groups to ensure they understood and were comfortable with the implementation procedure. Short checklists which outlined all of the steps that needed to be undertaken by WCOs were also shared. BIT team members were present at the WCU on the date of implementation, conducted visits throughout the trial, and were contactable to deal with any issues and questions raised throughout.

Both the WCU and BIT conducted monitoring throughout the trial. This included: carrying out one-on-one catch-ups with the WCOs in both the treatment and control groups to gauge their compliance with processes and understand their experiences of those in the treatment group in using the new approach; and running a follow-up training session before WCOs started to make reminder calls which included training on using the conversation guide and sending SMS reminders.

The IT system used by WCOs automatically logged when a new Warning Letter template was used. In addition, WCOs typically keep a record of communication with witnesses and victims in an electronic ‘Contact Log’. WCOs in the treatment group were also asked to make a record in the Contact Log when they used the new approach (such as the SMS reminder). However, this was not included in the statistical analysis presented below as WMP was not able to extract data from the Contact Log for the BIT evaluators.

As all WCOs worked in one open-plan office, there was a small risk of spillover, particularly with regard to using phrases mentioned in the new conversation guide. However, to minimise these effects, BIT circulated all staff guidance to remind the WCOs of the importance of maintaining treatment/control group practices (explicitly asking those in the control group not to adopt any of the new practices throughout the duration of the trial). The risk of spillover effects was believed to be minimal, due to the fact that WCOs rarely share cases (beyond job share partners and trainers/trainees, who were allocated to the same treatment or control group) and had little reason to discuss the specific processes used when interacting with witnesses and victims.

Although these adherence cues and measures were in place, one limitation of the approach was that it was not possible or practical to directly observe WCOs’ adherence to experimental protocols for each individual witness or victim. The analysis presented in the results is therefore “intention-to-treat”. Intention-to-treat (ITT) designs are common across the policy landscape; they are accepted as realistic as it is often not viable to measure activity with any precision. They also have the advantage that they acknowledge non-compliance, maintain sample size and allow for greater generalizability (Gupta 2011). Additionally, they allow the treatment and control group to have an assumed equivalence on all possible confounding factors prior to experimental intervention. ITT designs are the most conservative way to look for treatment effects in an RCT and hence a pragmatic indicator of effectiveness. Implicit in our analysis is the assumption that appropriate treatment was administered in all cases. This means that if WCOs assigned to the treatment approach did not follow it, or if WCOs assigned to the control approach used the treatment materials, our results would underestimate the effect of the treatment.

Statistical methods

In this section, we begin by presenting our pre-trial power calculations. Next, we describe the treatment and control samples and assess whether any differences between these groups could impact the interpretations of the analysis. We then look at experimental results for our primary outcome measure (attendance at court). Finally, we conduct subgroup analysis to look at the effects in our key subgroups of interest: victims vs. witnesses and domestic violence cases vs. all other cases.

Power calculations

We conducted power calculations to help inform the design of the trial, using historical data wherever possible. The measured parameters for our power calculations were: alpha = 0.05; beta = 0.80; intra-cluster correlation (ICC) = 0.05; number of clusters = 36; attendance in the control group = 84%; number of cases per cluster = 95. These calculations simulated running the trial for 5 months, assuming similar caseload volumes to those historically experienced by the WCU. Our power calculations suggested that the minimum detectable effect size (MDES) we could detect would be a 7.6% point increase in attendance. We predicted that by including covariates in the analysis, we could reduce the MDES by 25%, or to 5.7% points.

Achieving this effect size was determined as being fairly ambitious: if realised, it would be quite significant in real economic terms, reducing non-compliance by over 35% in relative terms. Unfortunately, there were limited ways that we could attempt to improve our statistical power. Running the trial for longer (e.g. involving more witnesses and victims over time) was one option, but the clustered nature of our trial and adjustments made for intra-cluster correlation mean that the returns of adding additional participants become more limited. Moreover, we used all eligible WCOs in the trial, so could not increase the number of clusters. Despite this fairly ambitious effect size target, we assessed it as being viable and proceeded with the trial.

Balancing treatment and control groups

To verify whether our treatment and control groups are balanced across relevant characteristics, we ran a series of balance checks. First, we compared characteristics of WCOs that are included in our analysis (N = 34). Next (and finally), we compared characteristics of witnesses and victims in our sample, as well as the characteristics of the WCOs they were assigned to as this is the level at which our analysis takes place. The justifications for the inclusion of the balancing characteristics chosen are as follows:

• WCO performance rating: we hypothesize that past performance of WCOs is indicative of their future performance.

• WCO pre-trial attendance rates: we hypothesize that WCOs with higher historic attendance rates (pre-trial) will be likely to have higher attendance rates during the trial.

• WCO experience: more experienced officers may be more effective; conversely, they may have more ingrained work practices and may be resistant to implementing the treatment.

• WCO work arrangement: there is variation in WCO working patterns, as some work part-time in job-share arrangements. It was therefore necessary to ensure that job-share partners were in the same group, as it is likely that they share cases (and we wanted to ensure that each witness or victim received a consistent approach).

• Domestic violence (DV) case: we hypothesize that the factors affecting DV victim attendance differ from those of other witnesses and victims (often the fear of reprisal); and baseline data suggested that non-attendance rates of DV victims were much higher.

• Gender of witness/victim: we hypothesize that there may be different factors affecting the likelihood that female/male witnesses or victims attend. For example, 89% of DV victims are female.

• Victim: we hypothesize that there may be different factors affecting the likelihood of victim and witness attendance (for example, witnesses may be further removed from the case).

• Ethnicity of witness/victim: we hypothesize that there may be different factors affecting the likelihood that witnesses or victims of different ethnicities/cultures attend.

• Key witness: being (and knowing that you are) a key witness will likely increase the odds that you will attend court.

• Age: we hypothesize that this will affect attendance. For example, certain age groups are more likely to be employed, which may make it more difficult to attend court. Conversely, elderly witnesses and victims may also struggle to arrange transportation.

Table 1 compares the WCO characteristics in the treatment and control groups (using 2015 data on witness and victim attendance) of the WCOs in our analysis sample. We present the mean of each characteristic by group and present the resulting p value of the difference, as measured by an Ordinary Least Squares (OLS) regressions. Note that two WCOs assigned to the treatment were on maternity leave and are therefore excluded from our analysis (and the balance checks presented below).

Table 1 WCO characteristic balance between the treatment and control groups

Table 1 demonstrates that groups were largely balanced against the amount each WCO works (i.e. full time, part time, job share), length of service (5 years +, 2 years, 18 months, 12 months), performance ratings from staff appraisals (i.e. good, satisfactory, unsatisfactory), and witness and victim attendance rates in the baseline data. We checked balance on both staff performance ratings and baseline attendance rates due to concerns that ‘high-performing’ staff members might have low baseline attendance rates, perhaps indicating that the baseline rates would not be good predictors of future attendance.

The pre-trial attendance rate for WCOs in the control group was greater (at p < 0.10) than the attendance rate for WCOs in the treatment. Given this difference, and the expectation that historical attendance rates are predictive of future attendance rates, we controlled for pre-trial attendance in our analysis.⁴

Table 1 also presents figures on the number of required attendances,⁵ which is a close proxy to the number of cases handled. The figures suggest that the treatment group dealt with slightly more cases on average, both in the pre-trial period as well as during the trial.⁶ However, these differences are not statistically significant.

Table 2 compares the characteristics of witnesses and victims in the treatment and control groups. Specifically, we present the mean of each characteristic by group, and present the resulting p-value of the difference, as measured by an ordinary least squares (OLS) regression (with clustered standard errors at the WCO level, our unit of randomisation).

Table 2 Witness and victim characteristic balance between the treatment and control groups

Examination of Table 2 reveals that one characteristic that differs between the treatment and control groups is the number of required attendances per witness or victim. The difference (0.039 attendances) is statistically significant, but its economic significance in terms of WCO caseload is likely limited. Still, we control for this characteristic in our secondary analysis in order to minimise any impact on those results.

Another significant difference is in the pre-trial attendance rates associated with witnesses’/victims’ WCOs. WCOs in the control group have significantly higher pre-trial attendance rates—we therefore control for this difference in our analysis. Otherwise, Table 2 suggests that the randomisation produced comparable groups of witnesses and victims.

As the majority of cases were finalised within a six-month period, the analysis below includes data from cases allocated to WCOs within the treatment period that were finalised before 6 January 2017. The final sample size was n = 2516 with n = 1345 in the treatment group and n = 1171 in the control group. It is worth noting that this analysis excludes 89 witnesses and victims in our trial (about 3.4% of the total potential sample), as the outcome data that we had for this group was aggregated at a level that prevented us from tracking individual-level behaviour.⁷

Statistical approach

The results presented below are generated from a series of Ordinary Least Squares (OLS) regressions. Standard errors are clustered at the WCO level (the unit of randomisation).

In our analysis, we run a series of models that omit covariates, but our preferred specification includes a number of characteristics of WCOs and witnesses/victims as covariates. This preferred model takes the following form:

$$Attended_{i} = \alpha + \beta_{1} Treatment_{i} + Characteristics_{i} \varGamma + \varepsilon_{i}$$

where $Attended_{i}$ is a binary variable that captures whether or not victim or witness i attended at least one hearing they were required to attend; $Treatment_{i}$ is a binary variable that captures whether or not victim or witness i was assigned to a witness care officer (WCO) assigned to the treatment; $Characteristics_{i}$ is a vector of covariates that includes:

• Whether the case was a domestic violence case (yes, no)

• Gender of the witness/victim (female, male, missing)

• Whether the individual was a victim (yes = a victim, no = a witness)

• Ethnicity of witness/victim (White British, not White British, not stated)

• Witness/victim is a key witness (yes, no, missing)

• Age bracket of witness/victim (18–24, 25–59, 60–69, 70–79, 80+, not provided)

• WCO pre-trial witness/victim attendance rates (continuous)

• WCO experience (less than 5 years, 5+ years)

• WCO work arrangement (full-time, part-time, job-share)

and $\varepsilon_{i}$ is a heteroskedasticity robust error term.

Using control variables in the regression analysis accounts for the variation between participants and their experiences with WCOs. Alternative approaches to dealing with variation exist, such as hierarchical models that account for clustering. However, the current analysis takes an informed approach on the variables that may impact on outcomes, and we therefore believe that potential systematic bias is appropriately addressed.

Results

Attendance

Figure 1 shows the comparative likelihood of witnesses and victims attending a trial after controlling for relevant covariates (detailed above). For completeness, Table 3 presents the result of the OLS in full (Column 3) and shows the effect of adding/removing covariates (Column 1 includes no covariates and Column 2 includes only those significant in Table 1). The analysis suggests that the treatment approach increased witness or victim attendance at a minimum of one hearing, but the increase within this trial is not statistically significant (p = 0.282). Specifically, after accounting for all control variables, witnesses and victims assigned to WCOs following the treatment approach are about 1.9% points likelier to attend at least one hearing than those assigned to WCOs following the standard approach. This effect would translate into an increase in attendance of approximately 48 witnesses or victims during our trial, if all witnesses and victims were treated.

Fig. 1

Likelihood of witness/victim attending at least one hearing

Table 3 Likelihood of witness/victim attending at least one hearing

A secondary question concerning outcome relates to the degree to which treatment from the WCO increases the likelihood that a witness or victim will attend all required hearings. This was because it was believed that witness and victim attendance is less likely in situations where witnesses and victims were required at multiple hearings (due to, for example, feelings of frustration with trial adjournments). Figure 2 shows that the treatment approach increases the likelihood of a witness or victim attending all required hearings by about 1.4% points (after accounting for all control variables). For completeness, Table 4 presents this result in full (Column 3) and shows the effect of adding/removing covariates (Column 1 includes no covariates and Column 2 includes only those significant in Table 2). If scaled to all witnesses and victims in our trial, this would have resulted in an increase of approximately 35 witnesses and victims attending all required hearings. Although this difference could be seen as encouraging, it is not statistically significant (p = 0.519) and hence we cannot conclude that the treatment has had an effect.

Fig. 2

Likelihood of witness/victim attending all required hearings

Table 4 Likelihood of witness/victim attending all required hearings

Subgroup analysis

As previously mentioned, it is useful to consider contextual factors that might make a difference to the outcome. It could be the case that the treatment is particularly effective (or ineffective) for specific groups or situations. Earlier we argued that two meaningful subgroups would compare results for domestic violence cases with all others and additionally compare the relative performance of the intervention for witnesses and victims. Table 5 reports the results of this analysis.

Table 5 Sub-group differences in outcome of intervention

In this case, our model takes the following form (using domestic violence as an example):

$$Attended_{i} = \alpha + \beta_{1} Treatment \,x\, DV_{i} + \beta_{2} Treatment \,x\ NonDV_{i} + \beta_{3}DV_{i} + Characteristics_{i} \varGamma + \varepsilon_{i}$$

where $Treatment \,x\, DV_{i}$ captures witnesses and victims of domestic violence cases assigned to the treatment; and $Treatment \,x\ NonDV_{i}$ captures witnesses and victims of another type of case assigned to the treatment.

The first two rows of Table 5 compare intervention effectiveness for victims and witnesses as separate groups. All else equal, victims in the treatment are about 2.8% points more likely to attend at least one hearing than victims in the control. This difference is not statistically significant (p = 0.194). For witnesses, there is virtually no difference between the treatment and control approaches (p = 0.826). Figures in the table also demonstrate that overall witnesses are more likely to attend a trial when requested compared to victims.

The second two rows of Table 5 examine the effectiveness of the treatment approach for domestic violence cases in comparison to those for other types of crime. The table shows that for domestic violence cases, the treatment approach increases the likelihood that a witness or victim appears by about 3.2% points. However, this difference is not statistically significant (p = 0.261). For non-domestic violence cases, there is virtually no difference between the treatment and control approaches (p = 0.774). Figures in the table also demonstrate that overall, witnesses and victims are more likely to attend non-domestic violence cases than domestic violence ones.

Whilst acknowledging that examining subgroups reduces the number of cases available for analysis, we also looked at the difference in control and treatment group outcomes for domestic violence victims only. These were identified as a particularly vulnerable group in the trial process and were nearly twice as likely to fail to appear than witnesses and victims associated with other cases. We therefore thought it was helpful to examine whether the behavioural insight approach was effective at engaging those that had very high stakes in the outcome of trials but obviously were likely to affect their social relationships as a result of the process. The final row of Table 5 demonstrates that overall, victims in domestic violence cases are the least likely of all sub-groups to attend trial. Results also demonstrate that for victims in domestic violence cases, the treatment approach increases the likelihood of appearance by about 2.7% points, but this is not a significant effect (p = 0.38).

Discussion

Victim and witness attendance at trial is a critical factor in an efficient and fair criminal justice system. It has been acknowledged that non-attendance contributes towards ‘cracked’ trials and has large resource implications in terms of ineffective public spending. There are a number of different strategies that might be used to encourage attendance, from the more coercive (threatening fines or other penalties) to more persuasive approaches. There is also a distinction between interventions that are designed to have immediate effect on victim/witness behaviour and those that are focused on longer term changes in court processes such as changes to procedures or training of court professionals.

This paper has taken a nudge perspective in designing an intervention to encourage victim and witness attendance at court. A significant feature of the design is that it was informed through examination of the possible mechanisms through which the treatment might change behaviour and consequently lead to the desired outcome (attendance at court). It has been argued that this practice is a feature of a strong evaluation design (e.g. Johnson et al. 2015). In particular, the design focused on the role of intentions, salience, reciprocity, social norms and timeliness in developing an effective communication strategy.

The resulting strategy was a three-pronged approach that encouraged attendance through (1) a new conversation guide; (2) a redesigned warning letter and (3) a new reminder call and SMS message. Given the multifaceted nature of our intervention, and the fact that the intervention involved training on-the-job, it was decided that the unit of analysis for administering the treatment would need to be at the Witness Care Officer level. Hence WCOs were randomly assigned to treatment or control and all their cases were expected to be dealt with in the same condition. This differs from the approach taken by Cumberbatch and Barnes (2017), who randomised at the individual witness/victim level and only administered a single intervention in the form of a SMS message.

The findings from this trial suggest that whilst interventions designed to improve attendance at Magistrates’ Court proceedings may prompt a positive change in witness and victim behaviour (in terms of attendance at court), the results are not statistically significant. In all cases and conditions, the treatment group showed a larger percentage attendance at court than the control, but these were often very close in real terms. It is interesting to note that whilst differences were not significant, the gap in outcomes between treatment and control groups tended to be greater for victims and for domestic violence cases. It is also worth noting that these groups are among the least likely types of witnesses and victims to attend court. Data from this trial suggests that victims in domestic violence cases may have particular reservations about attending court, which makes sense given that the stakes are likely to be higher for them, as they might face considerable life changes on the basis of the outcome in court.

The overall conclusion is that the intervention tested in this trial had no significant impact on levels of attendance at court. This result is in line with findings from Cumberbatch and Barnes (2017), which suggested that the nudge approach was not sufficient in influencing court attendance. There are a number of well-documented reasons for failure of interventions to achieve their outcomes that have been acknowledged in the evaluation literature (e.g. Knutsson and Clarke 2006). These include statistical measurement bias, low internal validity of the experiment, theory failure and implementation failure. The mechanisms for this intervention were fully considered upfront which guards against the possibility of theory failure. In this case, given the detailed set up of the randomised experiment, internal validity is also assessed to have not been the major cause of failure. However, we explore below the implications of our smaller sample size, which led to a less powerful experiment than anticipated. Further we explore the potential for implementation failure in terms of the actions of the WCOs. Finally, it could be that the activity did not trigger the mechanisms and lead to behaviour change as expected. We now move on to discuss some of these points in more depth.

A constraining factor of this particular trial was the unexpectedly low sample size. Judging from historical data, we anticipated having approximately 3767 witnesses and victims in our sample, whereas the intervention period itself yielded only 2516 witnesses and victims (only two-thirds the expected number). We were unable to increase our sample size by running the trial for longer, given agreements with our implementing partners to test the interventions for a limited time period. Moreover, running the trial for a slightly longer period of time would not have markedly improved our statistical power, given the extent of the intra-cluster correlation in terms of our outcome measures.

Our smaller-than-expected sample size ultimately meant that the experiment was potentially under-powered to detect a significant effect—particularly within sub-groups. Further research involving a larger sample of witnesses and victims would help us understand the true effect of this intervention, and it could be an approach that would be usefully replicated elsewhere in a bid to increase external validity.

Several steps were taken to ensure smooth implementation of the intervention. However, one of the additional potential trial constraints was the frequency with which WCOs picked up actions on each others’ work, throughout the duration of a case. This may potentially have led to spillover (or dilution) of the treatment approach. It is important to note that WCOs only shared cases with those in the same group (i.e. WCOs in the treatment group with others in the treatment groups and those in the control with others in the control). Contamination is common in crime prevention initiatives, as intervention relies on the actions of those implementing and is in contrast to, for example, medical trials where the same product is applied to all and participants undertake the trial independently of each other. Sampson (2010) correctly highlights concerns under the stable unit treatment value assumption (SUTVA) of RTCs. Since the WCOs were working alongside one another, there is a likelihood that the groups provided a less-than-stable treatment value, not only across officers but additionally over time if, for example, the officers talked to one another about how they were working. As described above measures were taken to avoid these inconsistent treatment- and spillover-effects. However, given the intention to treat design, the exact extent to which these happened in reality remains unknown.

To gather recommendations for future interventions we undertook a feedback survey on WCOs following the end of the trial. A couple of interesting suggestions were raised that could also be considered for future implementations of behavioural insight approaches to encourage court attendance. The first was to consider the impact of procedural justice (e.g. Mazerolle et al. 2013). Bottoms and Tankebe (2012) define this in the context of criminal justice as the ‘quality of decision-making procedures and fairness in the way citizens are personally treated by law enforcement officials’ p119. They suggest that legitimacy is dialogic and involves claims to legitimacy by power holders and responses by audiences. One example of a relevant element of legitimacy in this context is in terms of communicating expectations: it is important to inform the witness or victim from the start that they will be expected to attend court if they make a statement to the police.

The second suggestion that came from our WCO survey was to explore vulnerable sub-groups and to perhaps more specifically focus on the impact of different styles of interventions on attendance rates of victims of domestic violence. Indeed, there is a need to consider the extent to which nudge-based approaches such as those taken here address the entire ‘target’ population. It is acknowledged that these strategies are more likely to work on fairly typical populations (i.e. those ‘under the middle of the curve’) than on the particularly hard-to-reach groups. The strategies taken here can be seen as fairly ‘light touch’ from a nudge perspective and hence are less likely to be persuasive to those that are very unlikely to make a court appearance. Having said that, the degree of intervention was more extensive than the basic text message approach taken by Cumberbatch and Barnes (2017), and we find that the largest measured effects of the treatment (though still not significant) come from these hardest-to-reach groups. Perhaps the nudge approach is insufficient to overcome the fears of those that feel particular vulnerable or overcome the reluctance of those that do not wish to attend. In these cases, other types of intervention might be more suitable.

Conclusion

This paper has examined the efficacy of a nudge-based approach to encouraging victim and witness attendance in court. Whilst results failed to demonstrate a significant increase in attendance, there appears to be some evidence that further testing of these approaches would be worthwhile. Some distinct advantages of nudge interventions are that they are strongly based on mechanistic theory: there are explicit explanations for why and how these approaches can change behaviour. Other advantages are that once set up, they are easy to administer, and effects should be fairly immediate. This means that they have the potential to be cost effective ways of reaching a large population. Whilst it is unrealistic to expect all witnesses and victims to change their behaviour in response to such interventions, those that do and ultimately assist with a conviction as a result will be of great benefit to the criminal justice system.

Appendices

Appendices

Appendix 1: Treatment approach-initial call script

Appendix 2: Treatment approach-revised warning letter

Appendix 2 continued: Treatment approach-pre-trial checklist

Appendix 3: Treatment approach-reminder call

Appendix 4: Treatment approach-reminder SMS

Hi [insert first name or title/surname-up to WCO]. You are due to [insert court] on [insert date] at [insert time]. The Witness Service will be expecting you 30 min before to give you time to prepare. I’m here to help, so call me on [insert phone number] if you have any questions. Thank you, [insert WCO name].

Appendix 5: Control approach-Original Warning Letter