Measuring the Accessibility and Equality of Civil Justice

Abstract

While, in the UK, attitudes to the criminal justice system have been routinely investigated less effort has been made to measure attitudes to the civil justice system. However, globally, there are increasingly numerous examples of studies of attitudes to civil justice. Robust standardised scales are important to establishing change and difference between groups; for example, in the context of United Nations Sustainable Development Goal 16.3. Past focus has been on developing ‘composite indicators’. This paper sees the application of modern psychometric methods to the construction of ‘reflective’ instruments to measure single unidimensional attitude dimensions. Drawing on relevant theoretical frameworks, we administered an item pool of 35 attitude questions to a sample of 1061 adults across England and Wales. Principal component analysis was used to identify attitude domains, followed by Rasch analysis to construct scales with acceptable psychometric properties. An Inaccessibility of Justice (IOJ) scale and a Perceived Inequality of Justice (PIJ) scale resulted. Respondents who regarded themselves as having handled past civil legal problems poorly or having experienced unfair outcomes had higher IOJ and PIJ scores. Negative accounts of lawyers or courts from others were also associated with higher scores. Our substantive findings help explain how negative experiences of problem resolution can result in ‘frustrated resignation’ for later problems, and show the importance of positive experiences and accounts of the justice system in maintaining public support. The scales have broad justice policy utility and the methods provide a template for future related scale development.

1 Introduction

1.1 Background

A 2012 UK review of literature on public attitudes to the justice system found that, while attitudes to the police and criminal justice system have been routinely investigated in a reasonably nuanced manner, research into attitudes to the civil justice system “is close to non-existent” (Wilson 2012: 6). As Moorhead et al. (2008: 2) earlier observed, “the evidence base is dominated by studies either wholly in the criminal sphere or which fail to disaggregate civil and criminal contexts”.

The conflation of civil and criminal justice is a matter of some significance. The philosophical underpinnings, procedural norms, evidential requirements and extent of civil and criminal justice are largely distinct,¹ and “every society sufficiently developed to have a formal legal system uses the criminal-civil distinction as an organising principle” (Robinson 1996: 202). Thus, conflation reduces clarity of the subject and purpose of questioning, along with the meaning and utility of responses. Moreover, given that, in some jurisdictions at least, people’s conception of “the criminal and civil justice systems are one and the same” (Greene 2016: 1263), and “if left to define ‘the justice system’ for themselves the public do so in terms of criminal justice” (Stratton and Lowe 2005, p. 5),² conflation may well result in responses skewed to perceptions of the criminal justice system.

Reflecting Moorhead’s observation, the British Social Attitudes Survey has asked numerous questions concerning attitudes towards different aspects of policing and criminal justice, a few about confidence and trust in ‘the courts’ or ‘legal system’ in general, but almost none specifically about attitudes to aspects of the civil justice system. This despite the fact that we live in a “law thick” world (Hadfield 2010:133) in which civil legal problems³ have repeatedly been found to be “ubiquitous” (Genn 1999: 249, Pleasence et al. 2013, 2018).

Notwithstanding the above, some UK surveys have included questions seeking to gauge attitudes to particular facets of the civil justice system; including, with some irony, the Crime Survey for England and Wales, which has included questions concerning confidence in family courts in relation to child protection matters (Summerfield and Freeman 2014). Also, 9 of 11 national civil justice focused ‘legal needs’ surveys conducted across the UK over the past 25 years have included a small number of civil justice contextualised attitude questions (Genn 1999; Genn and Paterson 2001; Pleasence et al. 2004a; Dignan 2006; Pleasence 2006; Pleasence et al. 2004b; Pleasence and Balmer 2013; Franklyn et al. 2017).

Conflation of civil and criminal justice has also been common in investigations into public attitudes towards justice systems elsewhere in the world. For example, large scale broad based surveys such as the World Values Survey, the European Social Survey and Eurobarometer have routinely asked about confidence or trust in ‘the legal system’ or ‘courts’, without further specification of the aspects of the legal system or nature of the courts in question.⁴ Even dedicated surveys of justice or court systems, such as the many national and state surveys conducted in the United States, have often made no distinction between civil and criminal justice, or only rarely explicitly specified a civil context.⁵

However, globally, there are increasingly numerous examples of surveys that have incorporated investigations into public attitudes towards civil justice systems; albeit not always entirely separately from the criminal justice system.⁶ Most of the more than 50 national legal needs surveys undertaken over the past 25 years have explored attitudes towards formal civil justice systems, and civil justice more broadly, as have many of the larger number of sub-national surveys undertaken over the same period (Pleasence et al. 2018).⁷ For example, the various iterations of HiiL’s Justice Needs and Satisfaction Surveys have asked about perceptions of the quality of and trust in justice institutions, expectations of fairness in specific civil legal contexts, as well as perceptions of the quality of civil dispute resolution procedures and outcomes personally experienced by respondents (Pleasence et al. 2018).⁸ More broadly, many population survey based ‘access to justice assessments’, undertaken in a development (and legal empowerment⁹) context, have incorporated (sometimes extensive) sets of questions to expressly gauge attitudes to different aspects of civil justice systems, to “capture perspectives of the people on the ground, particularly those who are disadvantaged, in order to help shape policy and programmes” (UNDP 2012:2).¹⁰ Questions in such surveys have related to issues such as accessibility (e.g. geographical, financial, etc.) consistency, corruption, effectiveness, enforcement, equality, fairness, impartiality, respect, speed and trust (e.g. UNDP 2012).

Within the development context—in which recourse is often made to informal justice mechanisms—public trust in civil justice systems and the effective rule of law have been argued to be “a fundamental prerequisite for sustainable development” (Zoellick 2008: 75). This ‘economic development paradigm’ of justice reform sees effective civil justice systems to be not only a good in themselves, but also instrumental in realising “a range of … development goals: without justice, people cannot easily receive or access public goods and basic services, nor can they effectively access a range of markets” (Desai et al. 2012: 60). Moreover, a parallel ‘state-building paradigm’ of justice reform sees recognised and effective justice systems as “a sine qua non for peace” in fragile and conflict-affected states (Desai et al. 2012: 58). Thus, interest in assessing public attitudes towards justice institutions has also been a feature of rule of law and access to justice research in many Latin American countries with (relatively) recent experience of internal conflict, and where “during the transition from military and authoritarian to democratic regimes since the early nineties, the judiciary has been a central part of development and consolidation of the democratic process” (Cunha et al. 2014: 448).

Most extensively, civil justice features as the seventh of eight factors that comprise the World Justice Project’s Rule of Law Index. The civil justice factor, while mostly compiled from the qualified respondent questionnaires (sent to experts in each of the countries surveyed), also draws on three questions from the General Population Poll, which from 2016 has also incorporated a dispute resolution module (Adams et al. 2017). And, of course, there are also various, contrasting, examples of research into particular attitudes to particular aspects of particular civil justice systems in a range of jurisdictions; such as Quintanilla and Avtgis’s (2017) study of attitudes to pre-dispute binding arbitration clauses in the USA.

Some investigations of attitudes to civil justice have involved the development of standardised scales. For example, the World Justice Project’s Rule of Law Index represents a broad-based scale (and sub-scales) that incorporates such attitudes, as does Cunha et al.’s (2014) Brazilian Justice Confidence Index. There are also examples of much more narrowly focused scale development initiatives, such as Hans and Lofquist’s (1994) Litigation Crisis Scale—used to explore the causes and correlates of perceptions of the United States civil justice system being ‘in crisis’.

The rigorous development and evaluation of scales is important if change and difference between groups is of interest. It is also important to theoretical development in relation to such attitudes and, indirectly, impinges on related areas of research, such as into—to give two examples with significant bearing on contemporary access to justice policy—drivers of legal problem resolving behaviour or public perceptions relating to United Nations Sustainable Development Goals (SDGs) concerning the rule of law and access to justice (SDG 16.3) and the effectiveness, accountability and transparency of government institutions (SDG 16.6).¹¹

In the case of the first of these, the psychological mechanism through which people become inactive in the face of legal problems, through processes such as that described by Sandefur (2007, p. 125) as “frustrated resignation”—born of repeated “past experiences of frustration or failure in trying to resolve similar problems, even though the particular other party or parties involved had changed”—could be explored further through measures that could identify difference and change in attitudes to, say, the accessibility or fairness of dispute resolution processes. And, while there is some quantitative evidence that “people have a greater tendency to feel that the law and the justice system are essentially unfair if they had experienced justiciable problems, if they had experienced unfavourable outcomes to those problems and if they have had difficulty obtaining satisfactory assistance” (Currie 2009: 100),¹² opportunities for detailed and robust investigation to quantify and identify correlates of negative perceptions of different aspects of the civil justice system have been narrowed by the unavailability of appropriate research tools.

In the case of United Nations SDG 16.3, it is notable that, at present, the current global indicators framework includes two criminal justice, but no civil justice, related indicators. In part, this is down to limited availability of data and resources. The broad scope of civil law—touching many aspects of everyday life—along with the fact that only a minority of civil legal problems enter the formal justice system [the mean percentage of civil legal problems that saw court involvement, as reported by the World Justice Project’s 2016 General Population Poll, was only slightly above 10% (Adams et al. 2017)],¹³ means administrative data cannot provide a comprehensive picture of access to civil justice. Moreover, objective survey-based measures of access to civil justice are complex and relatively expensive to implement. Thus, relatively simple and cheap to implement survey-based measures of people’s perceptions of access to justice could be of particular interest in the SDG 16.3 context. Although, clearly many challenges present themselves in this regard. For example, people’s attitudes and wider perceptions are influenced “by a range of factors that may be unrelated to the performance of institutions” (Parsons 2011: 178), such as expectations and other social, cultural and historical factors. As Masterson and Tong (2015: 291) have observed,

“There is clear evidence that justice perceptions generally do not develop in a vacuum, but instead rely on social information and the broader social context. Whether the social information stems from other individuals such as peers or supervisors, or from a larger setting such as one’s work team or organisation, studies have shown that people’s individual justice perceptions are formed and influenced by one’s social setting”.

Moreover, access to justice can extend well beyond formal and state recognised justice systems, and measures in many, particularly developing, countries—along with global measures—must recognise and accommodate this. As the UNDP (2009: 8) have observed, “informal justice can form a key part of individuals’ and communities’ experience of justice and the rule of law, with over 80% of disputes resolved through informal justice mechanisms in some countries”. Furthermore, there are differences in designs and underlying goals of different justice systems around the world, which must also be recognised and accommodated in measurement design when appropriate.¹⁴

1.2 Attitudes to the Civil Justice System

Attitudes can be defined as “enduring pattern[s] of evaluative responses towards a person, object, or issue” (Colman 2015: 62). Attitudes are of interest not just in themselves, but also because they “provide a predisposition to behave in a particular way” (Coaley 2014: 231). Thus, an understanding of attitudes to the civil justice system is, as suggested above, an important element of understanding civil justice related behaviour.

Attitudes are multidimensional (i.e. they are made up of several domains). At the broadest level, they are commonly regarded as having three components; the affective, behavioural and cognitive (the ‘ABC’ components) (Rosenberg and Hovland 1960).¹⁵ The affective component refers to emotions or feelings. The behavioural component refers to behavioural intent. The cognitive component refers to beliefs. In the context of the civil justice system, these components can be distinguished in relation to, for instance, feelings, behavioural intent and beliefs concerning lawyers or civil courts, or other aspects of formal civil justice systems, which represent the scope of the present paper.¹⁶

Further dimensions of attitudes to civil justice systems have also been suggested. For example, Wilson’s (2012: 7) review identified “a range of types of attitude” based on different aspects of evaluation. Those she detailed referenced the “similar, but different” (Moorhead et al. 2008: 4) concepts of satisfaction, confidence, trust, support and legitimacy. Linked to these are also concepts such as interest in (or engagement with) the justice system, that are discussed within constitutional and access to justice debates, and which are also sometimes put forward as dimensions of attitude (e.g. Coaley 2014).

Klaming and Giesen’s (2008) and Verdonschot et al.’s (2008) extensive reviews of conceptualisations of the quality of justice procedures and outcomes, for the Hague Institute for Innovation of Law’s (HiiL) Measuring Access to Justice in a Globalising World project (Barendrecht et al. 2010),¹⁷ also listed a range of justice dimensions which can be the subject, and therefore further dimensions, of attitudes; namely, as regards process, procedural, interpersonal and informational justice and, as regards outcomes, distributive and restorative justice, along with outcome functionality and transparency.

Procedural justice refers to the various properties that a procedure should possess “in order to be perceived as fair by its user” (Klaming and Giesen 2008: 3), including “voice, neutrality, trustworthiness, consistency, and accuracy” (Gramatikov et al. 2011: 361). Interpersonal justice “reflects the degree to which people are treated with politeness, dignity, and respect by authorities and third parties involved in executing procedures or determining outcomes” (Colquitt et al. 2001:427). Informational justice is concerned with “explanations provided to people that convey information about why procedures were used in a certain way or why outcomes were distributed in a certain fashion” (Colquitt et al. 2001: 427). Distributive justice concerns the fair distribution of benefits and burdens (in this context, system outcomes),¹⁸ while restorative justice “is the dimension of the outcome which rectifies … damage or loss suffered” (Gramatikov et al. 2011: 363). Functionality of outcome “is the extent to which the outcome solves the problem”, and transparency of outcome concerns explanations for outcomes and the ability to compare the outcomes of similar problems (Gramatikov et al. 2011: 363).

Within these justice dimensions are apparent those that Wilson refers to as ‘neutrality’ (in terms of procedures and decision making), ‘treatment’ (in terms of respect), ‘having your say’ (i.e. the opportunity for parties to state their case) and ‘motive-based trust’ (i.e. the perceived motives of legal actors), associated with Tyler and colleagues’ influential theorising around procedural justice, legitimacy and compliance in the criminal sphere (e.g. Tyler 1990; Tyler and Huo 2002; Tyler et al. 2007).

Then there are dimensions of justice specific to different areas of the justice system (such as areas of law, forms of process, types of court, kinds of legal actor, etc.), which may extend beyond those just described, and which again can also be considered as dimensions of attitudes. So, for example, Moorhead et al. (2008) noted seven dimensions to measures of confidence in the criminal justice system within the British Crime Survey¹⁹; namely to: bring people who commit crimes to justice; meet the needs of victims of crime; respect the rights of people accused of committing a crime and treat them fairly; deal with cases promptly and efficiently; treat witnesses well; reduce crime; and, deal with young people accused of crime.

Broader socio-legal theory and research also points to a variety of further potential aspects of attitudes to civil justice.

For example, Ewick and Silbey’s (1998: 28) research in the field of legal consciousness suggests that, in the United States, “people tell three stories” about the law; as being ‘majestic’ (“operating by known and fixed rules in carefully delimited spheres”), ‘a game’ (“a terrain for tactical encounters through which people marshal a variety of social resources to achieve strategic goals”) or ‘arbitrary’ (“rather than objective, legality is understood to be arbitrary and capricious”). Each of these stories provides dimensions for attitudinal measurement.

Galanter’s (1974, 1999) theorising around the structural imbalance between the resources and prospects of ‘haves’ and ‘have-nots’ (and, closely linked to this, ‘one-shotters’ and ‘repeat players’) also highlights potential subjects and dimensions of attitude measurement; both in relation to the broad theme of the prospects of differentially resourced litigants and in relation to more narrowly formulated hypotheses—such as that one-shotter lawyers “tend to make up the ‘lower echelons’ of the legal profession” (Galanter 1974: 116), that the law is skewed towards repeat players, that lawyers improve case outcomes, and that the closer the ongoing relationship between parties to a dispute, the more likely “the relationship will be regulated by some independent ‘private’ system” (Galanter 1974: 130). Likewise, the newer literature focusing on the disadvantage of underprivileged repeat players does the same (e.g. Sandefur 2007; Green 2016).

Theorising around legal pluralism—or “the idea that in any one geographical space defined by the conventional boundaries of a nation state, there is more than one law or legal system … [or] normative orders or customs” (Davies 2010: 805/820)—raises issues around both commitment to competing normative orders (again, pertinent to attitudes to law) and the complex nature of attitudes in the many countries in which “legal pluralism is common” (Barendrecht 2011: 281).

In addition, policy debates concerning access to justice suggest further civil justice attitude dimensions. For example, the Bach Commission on Access to Justice’s “six key problems within the justice system” provide potential subjects for attitude measurement (Bach Commission on Access to Justice 2016: 8). Indeed, attitude to the cost of accessing the justice system—which features prominently within these problems—has been a routine subject of investigation in recent surveys (e.g. Franklyn et al. 2017).

In summary, the literature suggests a broad array of potential domains of access to justice; some theoretically well-developed, some supported by empirical investigation, some more conjectural.

1.3 Standardised Scales of Attitudes to the Civil Justice System

While, as was detailed in Sect. 1.1, examples exist of standardised scales of, or incorporating, attitudes towards aspects of civil justice systems, it is notable that they have tended to be composite indicators (i.e. the compilation of multiple individual indicators “into a single index” and measuring “multidimensional concepts which cannot be captured by a single indicator” (OECD 2008: 13). A significant body of work has examined the construction of composite indicators in legal (and related) contexts (e.g. Maggino 2017; Saisana et al. 2005; Saisana and Saltelli 2014) and detailed guidance on the construction of composite indicators has been published by the OECD (2008). However, less attention has been paid to the psychometric development of standardised measures of single unidimensional attitude dimensions.

A range of terminology has been used to distinguish between these two approaches, though perhaps the simplest is Stenner et al.’s (2009) distinction between index models (formative or composite variable models where the causal action flows from the indicators to the composite variable) and measurement models (reflective or latent variable models where the causal action flows from the latent variable to the indicators). Costa (2015) further formalises the distinction in quality of life research, between composite (or sometimes formative, where the construct is an exact linear combination of the indicators), causal (where the construct is an effect of the indicators) and reflective (manifestations of some underlying construct) approaches. Costa (2015: 2057) noted that “psychometric methods based on inter-item associations are only appropriate for reflective indicators, whereas other statistical and non-statistical validation methods can be used for composite or causal indicators”.²⁰

This study represents an early attempt to apply modern psychometric methods to the construction of reflective instruments in a legal context (and, specifically, in the context of attitudes to formal civil justice systems), utilising Rasch analysis, as described in the next section. As well as providing research tools in an area in which, as Stratton and Lowe (2005:3) have argued, many surveys “fail to meet most of the technical assumptions of polling”—often not defining their focus, or clearly explaining it to respondents—it is hoped that the methods set out in this paper will provide a template for the further development of similar and related standardised scales.

Moreover, as the OECD (2008: 18) have noted, modern psychometric methods can make a contribution to composite indicators, with a need further to explore “the relationship between the practice of composite indicators and the traditional measurement theory developed in psychometrics”. A number of authors, such as Gnaldi et al. (2017) have addressed this relationship through multidimensional extensions of Rasch/IRT approaches.

2 This Study

In this paper, we set out details of an application of modern psychometric methods to the construction of reflective instruments to measure attitudes to the civil justice system, principally within a UK context, but potentially of application in similar jurisdictions. The study has drawn on the principles and methods that have evolved in other disciplines (e.g. DeVellis 2012) and the latest developments in psychometric modelling (e.g. Hobart and Cano 2009; Christensen et al. 2013; Boone et al. 2014; Bond and Fox 2015). The study sought to first identify attitude dimensions, or domains, and then construct scales for measuring attitudes within these domains—with acceptable psychometric properties—utilising data from survey questions associated with individual domains. Finally, it sought to investigate the relationship between attitudes to civil justice and people’s experience of civil legal problems, dispute resolution, and social discourse on the same.

2.1 Methods

The development of measures, or scale development typically involves a number of set steps, as set out in DeVellis (2012).²¹ These include determination of what is to be to measured, generation of an ‘item pool’ (in this case, a list of statement-based attitude questions), administration of items to a development sample (in this case, via a general population probability survey conducted exclusively for the purposes of scale development), evaluation of item responses and optimisation of scale contents. The final two steps in this process typically involve either classical test theory (CTT) or modern psychometric methods (in this case, Rasch analysis).

2.1.1 Attitude Item Pool

As indicated above, the literature suggests manifold potential domains of attitude to the civil justice system; but, with limited space available within a survey questionnaire to devote to an attitude item pool, we were not able to explore each of these individually or in detail. So, instead, we sought to construct an item pool containing a broad and coherent range of aspects of attitudes, that might provide some indication of multidimensionality and domains, if there are distinct domains that are reflected in attitudes.

To inform the construction of the item pool, we collated relevant attitude questions from earlier surveys.²² There was little commentary on the origins or design of these earlier questions; a reflection of both the general absence of theoretical frameworks to guide formulation and of empirical testing. In many cases, questions appear to have simply reflected the prevailing interests of policy stakeholders; although in some instances questions were more considered, and some had empirical foundations. For example, the four attitude questions developed by Genn (1999: 225) “were largely drawn from comments made by members of the public during focus group discussions at the developmental stage of the research”.

In the end, questions from a variety of surveys were adapted for inclusion in the item pool²³; along with additional questions drafted to reflect key aspects of the dimensions outlines above. The text of the introduction to the item pool was informed by the framing of the family justice section of the Crime Survey for England and Wales; although it was broader in its scope:

Questionnaire introduction

Thank you for participating in this survey of people’s impressions and experience of justice and the justice system in England and Wales.

The survey is not about crime or criminal justice. Instead it is about the many other aspects of law and justice that touch people’s lives.

For example, people can face problems concerning employment, housing, children, money or injuries that have a legal dimension. When they do, they may ask lawyers (or other experts) to help sort such problems out. And if problems cannot be resolved between parties, then courts or tribunals (a less formal type of court) may also help achieve a resolution.

It is these types of problems and this system of experts and courts/tribunals that this survey concerns.

Some of the questions may not seem relevant to you. Do not worry. This is not a test. Just try to respond honestly and accurately.

All responses you provide are confidential.

Item pool introduction

As a reminder, we are not concerned with the criminal justice system. We are concerned with the justice system that deals with issues such as being unreasonably sacked by your employer, injured as a result of someone else’s negligence, involved in a dispute over money as part of a divorce, or facing eviction from your home

In total, the item pool comprised 35 questions, concerning law, courts/tribunals and lawyers. To support domain identification, they were focused on eight distinct theoretical dimensions of attitudes prominent within the literature and past surveys: outcome fairness; neutrality; manipulability; respect; voice; motivation; access/efficiency; sovereignty. All 35 items are set out in Table 1.

Table 1 Responses to the attitude to law items

2.1.2 General Population Survey

The survey used to collect item pool data utilised a one-stage sample design, in which a stratified, but unclustered, sample of addresses was drawn from the Residential Postcode Address File; the cornerstone of national probability samples in England and Wales. The survey was an innovative hybrid form of postal and online survey, based on the Community Life Survey web experiment.²⁴ The findings of the Community Life Survey web experiment suggest that the quality of data obtained from hybrid postal and online surveys can generally be expected to be similar to that which would have been obtained from face-to-face surveys.²⁵ Such hybrid surveys are relatively cheap to administer, but can maintain sampling rigour; unlike many pure online surveys.

Advance letters (and reminder letters and postcards) invited “the person aged 16 or over who has the next birthday” in a household to either take part in the survey online (using a provided web-link) or return a postal version of the questionnaire. The person completing the survey was offered a £10 shopping voucher as an incentive. In total, there were 1061 respondents, of whom 872 completed all the sections of the questionnaire.²⁶

In addition to attitude items, the survey also included a range of questions to explore legal confidence and capability, identify past interactions with and discussions about law, legal processes and legal services, and collect socio-demographic data.²⁷

2.2 Analysis

Scale development data analysis was conducted in two stages. First, principal components analysis (PCA) was used to explore the dimensionality of item responses and identify domains. Second, for those unidimensional domains that were identified, and associated with a suitable number of items, Rasch analysis was used to attempt to develop and refine scales with acceptable psychometric properties. Our use of PCA and Rasch analysis corresponds to Christensen’s guidance (Christensen et al. 2012). More generally, our approach to scale development broadly followed the guidance set out by DeVellis (2012).

For the two scales that this study yielded, a third stage of analysis was also undertaken. Three normal linear models were fitted to model scale score on the basis of recalled accounts of lawyers, courts and tribunals,²⁸ past experience (over the past 5 years) of civil legal problems, lawyers²⁹ or courts or tribunals.³⁰

2.2.1 Principal Components Analysis (PCA)

The aim of PCA was to establish whether responses to the attitude questions were multidimensional (i.e. were made up of more than one domains) and, if so, examine domains identified from the data. PCA allows identification of relatively independent subsets of items (Tabachnick and Fidell 2013).³¹ Subsets in which items are correlated with one another, but largely independent of those in other subsets, are combined into components, with the components thought to reflect underlying processes that created the correlations; in our case, attitude domains (Tabachnick and Fidell 2013). New components are derived in decreasing order of their importance. The first component accounts for as much variation in the original data as possible. The second component accounts for as much as possible of remaining variation; and so on (Everitt and Dunn 2013).

PCA was conducted using SPSS. Prior to analysis, data was confirmed to be suitable for PCA.³²

Analysis utilised principal components extraction, with oblique (direct oblimin) rotation.³³ The number of components retained was determined using examination of the scree plot (Cattell 1966), as well as parallel analysis (Horn 1965) using software developed by Watkins (2000).³⁴ These methods were used in favour of the commonly used Kaiser’s criterion (retaining components with eigenvalues above one) due to its inaccuracy (Velicer and Jackson 1990) and tendency to retain too many components (Costello and Osborne 2005).

2.2.2 Rasch Analysis³⁵

When solid components were identified using PCA, Rasch analysis was used to attempt to develop and refine scales; along with scoring protocols. Rasch analysis is a modern psychometric method widely used for scale development; and perhaps the only method specifically developed for this purpose (Smith et al. 2016).³⁶

The basic assumptions of the Rasch model are that: (a) each person is characterised by an ability, and (b) each item by a difficulty that (c) can be expressed by numbers along one line. Finally, (d) from the difference between the numbers (and nothing else), the probability of observing any particular scored response can be computed (Bond and Fox 2015). Equation (1) shows the basic Rasch model.³⁷

$${\text{The Rasch model}}:\quad p_{ni } = \frac{{e^{{(B_{n} - D_{i} )}} }}{{1 + e^{{(B_{n} - D_{i} )}} }}$$

(1)

where p_ni=the probability of affirming (i.e. giving a positive response), for item i and person n, D_i=the difficulty of item i, B_n=the ability of person n

Rasch analysis (using a partial credit model³⁸) was implemented using RUMM2030,³⁹ which allows assessment of a scale and its items against the Rasch model, as well as a detailed examination of scale functioning.

A number of measures of model fit were considered. Overall fit was assessed using an item-trait interaction statistic. This is reported in RUMM2030 as a Chi squared statistic and should be non-significant (following a Bonferonni correction for the number of items in the scale).⁴⁰ Two item-person interaction statistics were also considered (for items and persons). In each case, fit is represented by a z-score, where perfect fit would have a mean of zero and standard deviation of one. In practice, a fit residual value of 1.5 or less (for items and persons) was considered to indicate acceptable fit. Particularly in cases with misfitting items or persons, individual item and person fit was also explored using residuals and a Chi squared statistic (and Bonferonni adjusted p value). Items of concern were indicated by fit residuals below − 2.5 or above 2.5. Values below − 2.5 typically indicate overfit or redundancy, with the item a possible candidate for removal on this basis (i.e. it is already captured by other items). Values above 2.5 indicate a misfitting item. There could be a number of reasons for this, and a range of additional diagnostics allow further assessment, and possible correction of a misfitting item (see below). If an individual item fit cannot be improved, the item may need to be removed. As with items, misfitting persons were identified by fit residual values above 2.5.⁴¹

Internal consistency of a scale was assessed using the Person Separation Index (PSI), which gives a measure of a scale’s ability to discriminate between individuals with varying levels of the trait. Although acceptable levels may vary depending on the scale and its use, values exceeding 0.7 indicate acceptable internal consistency. Low values may indicate the need for additional items. Discrimination was also assessed graphically by examining the Item Characteristic Curve (ICC) for each item, with poor discrimination a common contributor to item misfit. The ICC allows examination of the extent to which an item deviates from the model; for example, by underestimating scores for low confidence groups or overestimating scores for high confidence groups.⁴² Remedial action may involve removing items with poor discrimination.

Item responses took the form of a four-point Likert scale. Suitability of the response format was checked by examination of the threshold map and category probability curves for individual items, which illustrates category structure. When individuals respond in a manner consistent with their level of the trait, thresholds should be ordered. Disordered thresholds can occur, for example, where there are too many response options or respondents struggle to differentiate between options. Remedial action may include rescoring response categories (i.e. to fewer categories).

Response dependence (or local dependence) was also tested as part of the Rasch analysis. Local independence is a requirement of the Rasch model and means that having extracted the Rasch factor (i.e. the attitude scale) there should be no leftover patterns in the residuals. Local dependence is an issue where response to one item depends on response to another item. A commonly cited example is that of numerous walking items in the same scale, where, if a person can walk a mile without difficulty, they must necessarily be able to walk a lesser distance without difficulty (Tennant and Conaghan 2007). Local dependence can inflate seeming reliability (indicated by an artificially high PSI) and affect parameter estimates in Rasch analysis (Tennant and Conaghan 2007). In practice, values over 0.2 in the residual correlation matrix indicate potential local dependence and redundancy (Marais & Andrich 2008). In such an instance, the first step is to carefully check the wording of the items. Sub-test analysis and re-running Rasch analysis having combined locally dependent items are means to assess the extent to which reliability has been inflated. Remedial action may involve the rewording or removal of items.

Rasch analysis requires that items form a unidimensional scale. While dimensionality was initially tested using PCA, it was also further explored as part of the Rasch analysis, using the procedure set out by Smith (2002). Following PCA of the residuals, correlations between items and the first residual components were used to define two subsets of items. An independent t-test was then used to test the difference in person estimates between the two subsets, with a non-significant result indicating no evidence of multidimensionality (i.e. indicating that the items measure a single attitude domain or trait).

Differential item functioning (DIF), which can have an impact on model fit, occurs when particular groups (e.g. men and women, younger and older respondents) perform differently on an item despite having comparable levels of the trait being measured. Graphically, DIF can be explored by superimposing groups of interest (e.g. men and women) on item characteristic curves to assess whether or not they perform differently. DIF is also assessed statistically using Analysis of Variance, to compare scores for each level of the person factor (men, women) and across levels of the trait (class intervals). There are two distinct forms of DIF (Teresi et al. 2000); Uniform DIF, where consistent systematic difference is observed in the groups’ responses to an item, would be indicated by a significant person factor. Remedial action here might involve separate calibration of the item for each group, though this has the disadvantage of adding complexity to the scoring of a scale. Non-uniform DIF, where differences vary across levels of the trait, would be indicated by class interval interaction. In this case, it is likely that the item would need to be removed.

Scale targeting was assessed graphically through examination of the person-item distribution, which illustrates individuals’ scores and item placement on the underlying trait. In a well targeted scale, items would span the full range of individual scores. This indicates that a scale is not too easy (e.g. attitude items that nearly all respondents would be positive about) and not too hard (e.g. attitude items that nearly all respondents would be negative about). Given that the scale is centred on zero logits, targeting can also be examined by how close the mean location value for persons is to zero. A poorly targeted scale is likely to indicate a need for additional easier or harder items to fully span individuals’ scores or replacement of items with similar difficulty to give a broader spread of difficulty.

Once a scale has been developed, satisfying the various requirements/diagnostics set out above, it can be used to produce a score. This requires providing guidance on how to calculate raw scores from responses, and a conversion table to change raw scores into Rasch converted scores. As Rasch location values have been converted from an ordinal to an interval scale (Wright and Linacre 1989), they are appropriate for a wider range of common statistical analyses. For ease of use, Rasch location values are also commonly converted to alternative ranges (e.g. 0–100).

Alternatively, scale scores can be converted into strata (Fisher 1992; Linacre 2013; Wright and Masters 2002), with the number of strata possible dependent upon internal consistency.⁴³

2.2.3 Normal Linear Models

Three normal linear models were fitted for each of the two scales that this study yielded. The first modelled scale score on the basis of recalled accounts of lawyers, courts and tribunals, whether civil legal problems had been experienced, lawyer use (in the past 5 years) and contact with courts or tribunals (in the past 5 years). The second replaced legal problem experience with a variable integrating self-reported adequacy of problem handling,⁴⁴ replaced lawyer use with a variable integrating satisfaction⁴⁵ and replaced contact with courts or tribunals with a variable integrating process fairness.⁴⁶ The third replaced self-reported adequacy of handling with perceived fairness of outcome within the legal problem experience hybrid variable.⁴⁷ In exploring relationships between scales and problem experience/interaction with the law, analysis also aims to explore construct validity.⁴⁸

3 Results

3.1 Responses to Items

Table 1 sets out responses to the 35 attitude items.⁴⁹

3.2 Attitude Domains

To identify attitude domains, the 35 attitude items were analysed using PCA. A significant Bartlett’s test indicated items were related ($\chi_{595}^{2}$ = 3931.23, p < 0.001). The majority of items had diagonal anti-image correlation matrix values above 0.7, further confirming suitability for PCA.⁵⁰ The correlation matrix also showed that all items, with the exception of items 13, 15, 27 and 30–33 had a correlation above 0.3 with at least one other item; again, confirming the factorability of the dataset/correlation matrix and its suitability for PCA.

PCA indicated that ten components had eigenvalues over one (Kaiser’s criterion). However, examination of the scree plot, while not decisive, indicated a slight drop-off after component six, and parallel analysis (Horn 1965) suggested retention of six components. As a result, six components were retained, explaining 47.6% of the variance. Direct oblimin rotation of the six-component solution yielded the pattern matrix shown in Table 2. Rotation showed a comparatively clear pattern, with the majority of items loading reasonably strongly on a single component, though there were ten items with loadings above 0.3 on two components. Correlation among components varied from 0.014 (between component 1 and 5) to 0.36 (between component 1 and 3),⁵¹ with clear multidimensionality and no case for combining components into a single score.

Table 2 Pattern matrix of the six-component solution of attitude to law pool items

On the basis of its content, component one was labelled ‘trust’. The second component was labelled ‘access’. The third component was labelled ‘equality’ and linked to the theoretical domains of outcome fairness and neutrality. The fourth component was labelled ‘fairness’. The sixth label was labelled ‘respect’. There were not enough coherent strongly loading items to label the fifth component. The majority of items also had reasonable loadings, with a minority falling below commonly used cut-offs.⁵² The ‘access’ component was the most coherent. Additional items would need to be tested to further explore other domains and sub-domains.

3.3 Development of Attitude Scales

While five coherent components were extracted using PCA, this did not mean that five scales could be developed using Rasch analysis. Initial Rasch analysis of the ‘trust’, ‘respect’ and ‘fairness’ components indicated that there were too few items to yield scales with acceptable psychometric properties. Specifically, in each case the person separation index fell below commonly used values indicating acceptable ability to discriminate between individuals with differing levels of each trait.⁵³ Larger item pools would have to be constructed and tested to develop functioning scales for these components. Nonetheless, of the components identified by PCA, two were suitable for scale development using Rasch analysis: ‘access’ and ‘equality’. Additional items would be required to develop ‘trust’, ‘respect’ and ‘fairness’ components into well-functioning rating scales.

3.3.1 An ‘Inaccessibility of Justice’ Scale

Rasch analysis of access items (the ten items with loadings above 0.3/below − 0.3 in component 2, Table 2) was restricted to those answering eight or more of the ten access items (446 of 535, 83.4%). The initial model (of ten items) was close to acceptable fit, with an item fit residual of 1.42 and person fit residual of 1.51, but with a significant item-trait interaction (indicating some deviation from the model⁵⁴). The initial PSI was 0.77. There was also some indication of disordered thresholds for item 8 and overfit/redundancy for a further item (item 21). Once response categories had been recoded for item 8 and the redundant item removed, all measures of fit were acceptable, with an item fit residual of 1.10, person fit residual of 1.40 and non-significant item-trait interaction.⁵⁵ There was no evidence of local dependence, multidimensionality or DIF on the basis of gender, age, legal problem experience or prior use of lawyers or courts, while the scale appeared to be reasonably well targeted (though a number of thresholds tended towards the lower end of the scale—Fig. 1).

Fig. 1

The person-item distribution graph for the inaccessibility of justice items

The final nine-items yielded a scale with acceptable psychometric properties that can be used as a measure of the inaccessibility of justice.⁵⁶ The Inaccessibility of Justice (IOJ) scale is as follows (renumbering items from 1 to 9):

Now, some questions about your general impression and experience of the justice system.

We are not concerned with the criminal justice system. We are concerned with the justice system that deals with issues such as being unreasonably sacked by your employer, injured as a result of someone else’s negligence, involved in a dispute over money as part of a divorce, or facing eviction from your home.

Thinking about issues like this, to what extent do you agree or disagree with the following statements. (strongly agree; mainly agree; mainly disagree; strongly disagree)

Q1:

Issues like these are usually resolved promptly and efficiently

Q2:

People with less money generally get a worse outcome

Q3:

For issues like these, law is like a game in which the skilful and resourceful are more likely to get what they want

Q4:

It is easy to take issues like these to court if needed

Q5:

For issues like these, lawyers are too expensive for most people to use

Q6:

The justice system provides good value for money

Q7:

For issues like these, people like me can afford help from a lawyer

Q8:

Rich people’s lawyers are no better than poor people’s lawyers

Q9:

Taking a case to court is generally more trouble than it is worth

After administration, responses are scored to yield, first, a ‘raw’ score, then a Rasch converted ‘IOJ score’. To calculate the raw score, for items 1, 4, 6, 7 and 8, strongly agree should be assigned a score of ‘0’, mainly agree ‘1’, mainly disagree ‘2’ and strongly disagree ‘3’. For items 2, 3 and 9, strongly agree should be assigned a score of ‘3’, mainly agree ‘2’, mainly disagree ‘1’ and strongly disagree ‘0’. For item 5, strongly agree should be assigned a score of ‘2’, mainly agree ‘1’, mainly disagree ‘0’ and strongly disagree also ‘0’. Across the nine items this will yield individual scores of between 0 and 26. These scores can be converted into IOJ scores (ranging from 0 to 100) using Table 3. Higher scores are associated with greater perceived inaccessibility of justice.

Table 3 Scoring for the nine-item ‘Inaccessibility of Justice’ (IOJ) Scale

The mean IOJ score among 357 survey respondents was 58.1 (standard deviation = 11.6). The median score was 45.2 (25th percentile = 32.8, 75th percentile = 60.3). Converting scores into strata (low, medium and high scores)⁵⁷ gave 26.1% with low scores (scores of 0–50), 54.1% with medium scores (scores of 51–66) and 19.9% with high scores (scores of 67–100). While based on relatively small numbers, these scores and strata provide initial baseline general population estimates for comparison.

The output of the three normal linear models of IOJ score is set out in Table 4. While there was no significant association between IOJ score and the simple experience of civil legal problems, once self-reported adequacy of problem handling and perceptions of outcome fairness were taken into account, the association became significant. Compared to those who had experienced no problems, respondents who handled one or more problems poorly were associated with a 9.5 point increase in IOJ score.⁵⁸ Similarly, compared to those who had experienced no problems, those who perceived one or more problems as having an unfair outcome were associated with a 6.4 point increase in IOJ score.⁵⁹ Accounts respondents had heard about lawyers, courts and tribunals from friends, family or colleagues were also all significantly associated with IOJ score. In all three cases there was a significant difference between those recalling positive accounts and those recalling negative accounts of 6.0 points for lawyers,⁶⁰ 5.4 points for courts⁶¹ and 4.9 points for tribunals.⁶²

Table 4 Three normal linear models of IOJ score on the basis of a range of variables relating to experience of law

3.3.2 A ‘Perceived Inequality of Justice’ Scale

Rasch analysis of equality items (component 3, Table 2) was restricted to those answering seven or more of the eight items (716 of 908 respondents, 78.9%). Inclusion of all eight items, gave an item fit just over acceptable limits (item fit residual of 1.54), an acceptable person fit (person fit residual of 1.39), but resulted in some deviation from the overall model.⁶³ The PSI was 0.76. There were no disordered thresholds, no local dependence, no evidence of multidimensionality and no misfitting individual items. However, there was strong evidence of uniform differential item functioning for item 34 (“For issues like these, if a man and a woman make identical claims in court, the man will be more likely to obtain a favourable result”) on the basis of gender (shown in Fig. 2). This suggested that male and female respondents interpreted the item (and its ‘difficulty’) differently.

Fig. 2

Differential item functioning for item 34 on the basis of gender

In the case of uniform DIF, remedial action can involve separate calibration of the item for each group (in this case for men and women); although this can introduce unwanted complexity into scale scoring and implementation. More generally, this case highlighted issues associated with items that refer to a specific demographic characteristic, such as gender, ethnicity or religion (rather than no background characteristics or general/multiple characteristics). Item 12 (“If a white person and a black person make identical claims in court, the white person will be more likely to obtain a favourable result”) might be expected to lead to similar issue were DIF tested on the basis of ethnicity. As a consequence, items 35 and 12 were both removed, leaving six equality items all of which referred to multiple characteristics or made no mention of background characteristics.

The six remaining items gave acceptable item (item fit residual of 1.40), person fit (person fit residual of 1.31) and overall fit.⁶⁴ The PSI of 0.74 was relatively low, but above the common cut-off of 0.7 and acceptable for group use. There was no evidence of DIF on the basis of gender, age, problem experience or prior lawyer/court use, evidence of multidimensionality or local dependence. All thresholds were ordered and the scale was reasonably well targeted (Fig. 3).

Fig. 3

The person-item distribution graph for the perceived inequality of justice items

The remaining six-items yielded a scale with acceptable psychometric properties that can be used as a measure of perceived inequality of justice. The Perceived Inequality of Justice (PIJ) scale is as follows (renumbering the items from 1 to 6):

Now, some questions about your general impression and experience of the justice system.

We are not concerned with the ‘criminal’ justice system. We are concerned with the justice system that deals with issues such as being unreasonably sacked by your employer, injured as a result of someone else’s negligence, involved in a dispute over money as part of a divorce, or facing eviction from your home.

Thinking about issues like this, to what extent do you agree or disagree with the following statements. (strongly agree; mainly agree; mainly disagree; strongly disagree)

Q1:

People with less money generally get a worse outcome

Q2:

For issues like these, law is like a game in which the skilful and resourceful are more likely to get what they want

Q3:

The law always treat both parties fairly, whatever their background, gender, ethnicity or faith

Q4:

Judges have their own agendas separate from the law

Q5:

The decisions and actions of courts are influenced by pressure from the press and politicians

Q6:

Courts and tribunals always treat both parties fairly, whatever their background, gender, ethnicity or faith

After administration, responses should be scored to yield, first, a ‘raw’ score, then a Rasch converted ‘PIJ score’. To calculate the raw score, for questions 1, 2, 4 and 5 responses of ‘strongly agree’ should be assigned a score of 3, ‘mainly agree’ a score of 2, ‘mainly disagree’ a score of 1 and ‘strongly disagree’ a score of 0. For questions 3 and 6 should be reverse scored with responses of ‘strongly disagree’ should be assigned a score of 3, ‘mainly disagree’ a score of 2, ‘mainly agree’ a score of 1 and ‘strongly agree’ a score of 0. Across the six items this will yield individual scores of between 0 and 18. These scores can be converted into PIJ scores for use in analysis (ranging from 0 to 100) using Table 5. Higher scores are associated with greater perceived inequality of justice.

Table 5 Scoring for the six-item ‘Perceived Inequality of Justice’ (PIJ) Scale

The mean PIJ score among 377 survey respondents was 53.0 (standard deviation = 15.7). The median score was 51.3 (25th percentile = 42.0, 75th percentile = 64.2). Converting scores into strata (low, medium and high scores) gave 27.3% with low scores (scores of 0–44), 53.8% with medium scores (scores of 45–66) and 18.8% with high scores (scores of 67–100). Again, while based on relatively small numbers, these scores and strata provide initial baseline general population estimates for comparison.

The output of the three normal linear models of PIJ score is set out in Table 6. There was a significant association between PIJ score and the experience of civil legal problems. Compared to those who had experienced no problems, respondents who reported problems were associated with a 3.4 point increase in PIJ score⁶⁵; with differences becoming more pronounced once self-reported adequacy of problem handling and perceived outcome fairness were included in analysis. Respondents who reported handling (one or more) problems poorly were associated with a 9.8 point increase in PIJ score,⁶⁶ and those who reported (one or more) unfair outcomes were associated with a 7.9 point increase.⁶⁷

Table 6 Three normal linear models of PIJ score on the basis of a range of variables relating to experience of law

While the number of respondents with experience of courts or tribunals was relatively small, there was a significant 9.0 point score difference between those who regarded processes they experienced as fair and those who regarded them as unfair.⁶⁸

There were also significant associations between PIJ score and accounts respondents had heard of lawyers and courts from friends, family or colleagues. In particular, positive accounts were associated with reductions in PIJ score. For accounts of lawyers, those recalling positive accounts scored 5.7 points lower than those not recalling an account⁶⁹ and 7.2 points lower than those reporting a negative account.⁷⁰ Similarly, for courts, those recalling positive accounts scored 6.5 points lower than those not recalling an account⁷¹ and 9.3 points lower than those reporting a negative account.⁷²

4 Discussion

While notable efforts have been made to develop composite indicators of, or incorporating, attitudes towards aspects of civil justice systems, less attention has been paid to the psychometric development of standardised measures of single attitude dimensions. The development of such measures would support theoretical development around attitudes to the civil justice system and enable reliable comparison of unidimensional scale estimates between population groups and over time. This would also benefit related research, such as into drivers of legal problem resolving behaviour or public perceptions relating to United Nations SDG 16.3 and 16.6.

Our study sought to apply modern psychometric methods to the construction of reflective instruments to measure attitudes to the civil justice system; principally within a UK context, but potentially of application in similar jurisdictions also. Our approach was firmly grounded in socio-legal theory, but also in the principles of scale development that have evolved in other disciplines and the latest developments in psychometric modelling.

Principal component analysis (PCA) of responses to civil justice system attitude item pool questions indicated five attitude domains. While these did not exactly mirror the eight theoretical domains that informed the design of the item pool, they nevertheless broadly linked to these theoretical domains and other domains suggested in the literature. The five empirically derived domains were labelled trust, access, equality, fairness and respect.

Rasch analysis allowed the development of two appropriately functioning civil justice attitude scales, utilising the items associated with the access and equality domains; yielding an Inaccessibility of Justice (IOJ) scale and a Perceived Inequality of Justice (PIJ) scale. Larger item pools would need to be constructed and tested to develop functioning ‘trust’, ‘fairness’ and ‘respect’ scales with acceptable psychometric properties.

Of the nine questions making up the IOJ scale, six directly concern resources. These link closely to the broad access/efficiency dimension of attitude prominent within the literature. More particularly, they link to Ewick and Silbey’s (1998) ‘game’ story of the law (especially Q3), Galanter’s (1974) theorising about the structural imbalance between the resources and prospects of ‘haves’ and ‘have-nots’ (especially Q2 and Q8), and contemporary debates around access to justice. The remaining questions directly concern efficiency.

Of the six questions making up the PIJ scale, two are also included in the IOJ scale. These concern equality of outcomes. However, the majority of the six questions that make up the PIJ scale concern procedural fairness and motive-based trust.

The findings from our analysis of the relationship between PIJ and IOJ scale scores and people’s experience of civil legal problems, dispute resolution and related social discourse were unambiguous, and have implications in relation to both the development of the literature and access to justice policy.

Sandefur (2007) has theorised that repeated experiences of failure in trying to resolve civil legal problems—such as among the low- and moderate-income residents of the Midwestern American city that she studied—can lead to “frustrated resignation” that manifests in increased inaction in the face of such problems. Survey findings, such as M/A/R/C research’s (1998) finding that negative experience of courts was associated with similar or lowered perceptions of courts, and Currie’s (2009) later finding that survey respondents were more likely to regard the civil justice system as unfair if they reported having experienced civil legal problems, unfavourable outcomes to those problems or difficulty obtaining satisfactory assistance point to changes in attitude that might underpin frustrated resignation.⁷³ More recently, Green’s (2016:1270) study of race, class and access to civil justice, has further indicated “connections between experiences with the criminal justice system and utilization of the civil justice system”. This “much broader connection between past experiences and decisions about whether to pursue resolving legal issues, even when past experiences have little or no relationship to the civil justice issue at hand” (1275) reflects both low levels of legal understanding of law and the justice system in general (e.g. Jamieson and Hennessy 2007; Pleasence et al. 2017) and the wide range of potential influences on attitudes to the civil justice system.

While indicating that experience of civil legal problems does not impact significantly on mean IOJ scale scores, our findings confirm that negative experience of problems—defined as self-reported inadequacy of problem handling or negative perceptions of outcome fairness—negatively impacts on IOJ scale scores. Those respondents who felt they handled problems poorly and those who perceived problems as having an unfair outcome were both associated with increases in IOJ score.

In the case of PIJ scale scores, a more general association between experience of civil legal problems and attitudes was also observed. Compared to those who experienced no problems, those who had experienced problems had higher PIJ scale scores. But, in line with IOJ scale score findings, self-reported inadequacy of problem handling and negative perceptions of outcome fairness both pushed scores higher still.

Going further than personal experience, both IOJ and PIJ scale scores were also associated with accounts respondents had heard of lawyers and courts from friends, family and colleagues. Negative accounts of lawyers and courts were both associated with increases in IOJ and PIJ scale scores.

In general, the impact of negative experience and social accounting was to some extent counterbalanced by a positive impact of positive experience and social accounting.

These substantive findings are important in the context of the developing literature, as well as access to justice policy. They support and add further detail to Sandefur’s (2007) theory of behaviour change, as well as giving insight into the broader range of influences on attitudes to the civil justice system. They also show that the value of supporting individuals through dispute resolution processes is manifold; extending not only to improving access to justice in relation to individual cases, but also to maintaining the integrity of the civil justice system and increasing the propensity of individuals, and those to who they relate their experience, to appropriately address future civil legal problems. Thus, policymakers should make all practicable efforts to promote positive experiences of dispute resolution. Of course, the present study was designed to facilitate scale development and did not contain the level of detail on justiciable problem experience found in legal need surveys. Integration of scales into future legal need surveys would allow far greater insight into the relationship between attitudes to justice and problem experience, response to problems and problem outcome.

With access to justice policy increasingly informed by an improving understanding of the public’s experience of law and dispute resolution (e.g. OECD 2017), legal professionals and justice institutions are well placed to utilise current understanding, as well as customised user feedback, to improve access to their services and the experience of users. However, the world over, the majority of dispute resolution occurs beyond the purview of formal dispute resolution authorities and legal professionals (Pleasence et al. 2018), particularly in those countries in which informal justice is particularly prevalent. So—particularly given the impact of social accounts of dispute resolution on civil justice system attitudes, and likely impact of social discourse and media representations more generally—the challenge of improving general public attitudes to the civil justice system is a substantial and complex one.

Finally, links between experience, attitudes and behaviour give measures of attitudes to the civil justice system broad relevance to policy objectives such as United Nations SDG 16.3 and 16.6; although, evidently, the challenges of designing globally valid attitude scales in the civil justice sphere are substantial. Nevertheless, our findings suggest that public perceptions of the justice system, as well as being potential proxies for accessibility and effective functioning, also constitute drivers of accessibility. Thus, measures of attitudes to the civil justice system—such as those measured by the IOJ and PIJ scales—which allow determination of levels and changes in levels of attitudes across and within populations can, with relative ease, provide both indication of system functioning and flagging of problems and populations that warrant investigation and policy intervention.

5 Conclusion

While a much broader range of item pool questions will need to be tested to develop further unidimensional attitude scales in the context of the civil justice system, our findings provide new insight and two theoretically coherent and robust attitude scales. Our approach—drawing on the latest developments in psychometric modelling—also provides a template for the development of similar scales in the future.

Our substantive findings provide further evidence of a psychological process through which negative experience of civil justice problem dispute resolution transforms attitudes in a manner that can manifest in frustrated resignation in the face of later legal problems. Moreover, our findings suggest that, in addition to personal experience, social reporting of the experience of others can bring about the same attitude changes. These findings have clear policy implications, both in relation to supporting individuals through dispute resolution processes and in relation to public legal education to counteract the impact of negative social commentary on the civil justice system. However, while improving the experience of clients and better supporting individuals engaged in formal dispute resolution processes are substantially within the control of legal professionals and justice institutions—and can be informed by increased understanding of the ‘user’ perspective of such processes—counteracting social discourse on civil justice systems may present a greater challenge, as this may well be bound up with wider representations of justice systems that are poorly understood by populations.