Is a fine still a price? Replication as robustness in empirical legal studies

https://doi.org/10.1016/j.irle.2020.105906Get rights and content

Abstract

Can fines lead to more of an undesirable behavior, rather than deterring it? This was the surprising finding in Uri Gneezy and Aldo Rustichini, “A Fine is a Price” published in the Journal of Legal Studies in 2000. In this field experiment at Israeli daycares, the introduction of fines caused an increase in late pick-ups by parents. The study has been frequently cited, especially for its suggestion that a fine can act as a price for non-compliance that “crowds-out” social norms and motivations for individuals.

In this article, we conduct two related studies to explore the robustness of Gneezy & Rustichini’s findings and the relevance of their suggested explanations. We seek to replicate their results using experimental surveys administered on MTurk, an increasingly common methodology in empirical legal studies, psychology and economics. While not an exact replication, it allows us to control aspects of the experimental design that are difficult to replicate in the field. We are also able to directly investigate whether fines persistently change the way respondents perceive the consequences (signaling, completing the contract) or relevant social motivations (crowding-out), as suggested by Gneezy & Rustichini. In the first study we translate the original daycare field setting into a vignette-based experimental survey. Our second study similarly investigates the effect of introducing a fine on income tax reporting compliance – an example suggested in the original study. In both studies, respondents are randomized into experimental conditions exposing them to one of two alternate fines or a social norm-based measure. We solicit multiple compliance measures for respondents along with measures of the importance of the alternate explanations to their decisions.

Our survey results do not replicate the original findings. In both our daycare and tax studies, the introduction of fines causes respondents to reduce non-compliant behaviour. Respondents expect others to behave similarly. Fines do not cause respondents to adjust their concerns about an incomplete contract consistently with Gneezy & Rustichini’s theory. They also show very little evidence of fines crowding out social motivations, despite being responsive to our social treatments. The effects of fines on outcome behaviours and respondents’ reasons are transitory. Once the fines are removed, our respondents return to their baseline behaviours. The survey results are consistent with intuitive judgments (and standard rational-choice theory) that fines deter. A survey is not a field experiment; however, our results suggest that more research is required to understand when and how any “fine is a price” effect may arise.

Introduction

Uri Gneezy and Aldo Rustichini published “A Fine is a Price” in the Journal of Legal Studies in 2000.1 The authors used a randomized field experiment at Israeli daycares to test whether “when negative consequences are imposed on a behaviour, they will produce a reduction of that particular response.” Instead of validating this standard assumption in psychology, economics, and legal studies, the authors found that introducing a fine increased late behaviour in their experiment.

Gneezy & Rustichini offered two possible explanations – both of which involved the fine transforming parents’ perceptions of the context. The first explanation was that the fine provided information about the consequences of coming late that had been unspecified in the contract between parents and the daycare. While parents may have previously feared severe consequences from coming late, introduction of the fine made it clear this was not a threat.

Gneezy and Rustichini offered another powerful explanation: before the fine existed, a social norm restrained parents from coming late. Once the fine was introduced, it transformed the parents’ perceptions of the extra care into a commodity. They were now free to consume it at the “price” of the fine, without the guilt or shame felt when transgressing the social norm. Parents’ internal, moral motivations were “crowded out” by the market signal sent by the fine. Gneezy & Rustichini’s experiment has been widely accepted as support for this latter, intuitively appealing theory. The paper has been cited more than 2000 times including in popular non-fiction for general audiences.2

Gneezy and Rustichini’s social norms explanation fit well with emerging developments in economic theory that expanded standard approaches to individual preferences. Theorists developed models for individual preferences that reflected altruism and other-regarding behaviour, drawing on empirical support from standard economic games.3 Another strand of emerging theory developed models of preferences dependent on reference points, providing a related theory of how a shift in behaviour might occur.4 Scholars also provided models and empirical evidence consistent with individuals behaving differently in contexts perceived as social as opposed to market exchanges.5

Despite these developments, questions remain about the extent to which social norms influence choices in economic environments and whether fines “crowd out” the effects of norms, as Gneezy and Rustichini suggested.6 Most relevant for our project, there has been some criticism of Gneezy and Ristichini’s original field study7 and the authors themselves suggested potential benefits from replication, even undertaking their own follow-up study.8

Given the significant influence of the original paper and the importance of the prospective behavioral insight it uncovers, we believe it is a good candidate for a form of replication study. However, in the case of field studies like Gneezy and Rustichini’s original, faithfully reproducing the full set of underlying conditions embedded in the social and temporal context for the field experiment is challenging. This would make it difficult to view any inconsistent result as a true failure to replicate. For empirical work that is inherently contingent on complex background factors such as legal institutions, culture and political conditions, we suggest that a “many studies”9 approach as a way to test robustness of the underlying mechanism may be more appropriate than strict replication. Empirical work and results like this are perhaps best conceptualized as an ongoing project that inherently involves the collaboration of a community of researchers. The creation of a body of related work will ground confidence in the insights and an understanding of their limits.

With this approach in mind, we conduct a number of related studies to explore the robustness of Gneezy & Rustichini’s original results. We transform the original daycare field setting into a vignette-based experimental survey administered on Amazon’s Mechanical Turk (MTurk) platform. This approach allows us to control aspects of the experimental design that are difficult to replicate in the field, and to directly investigate the alternative explanations Gneezy and Rustichini offered for their original results. We also test for evidence of the underlying mechanisms they identify in a tax compliance setting – an alternate context where Gneezy and Rusitchini suggested the “fine is a price” effect could appear. Survey-based research in empirical legal studies has grown, particularly as accessibility through online platforms such as MTurk has expanded.10 We seek to replicate Gneezy & Rustichini’s famous result with this increasingly common approach. While our study is thus not a direct opportunity to confirm or refute the original field trial findings, we hope to provide evidence relevant to their generality and contribute to the emerging literature employing MTurk in empirical legal studies, psychology and economics.

Our survey results do not replicate the original findings. In our daycare and tax studies, the introduction of fines causes a reduction in anticipated non-compliant behaviour by our survey respondents. Moreover, subjects anticipate that other respondents will be similarly affected. Fines do not cause subjects to adjust their concerns about an incomplete contract in a manner that is consistent with Gneezy & Rustichini’s theory. They also show very little evidence of fines crowding out any social norms. Moreover, the effect of fines on outcome behaviours and the reasons behind the outcomes of our respondents is transitory. Once the fines are removed, they return to their baseline behaviours. These survey results are consistent with our intuitive judgments (and standard rational-choice expectations) that fines deter.

The remainder of the paper is organized as follows. In Section 2 we introduce our primary replication of Gneezy and Rustichini’s daycare study. Section 2 includes discussion of our experimental structure and survey instrument, survey implementation and data collection, and results. In Section 3 we extend the “Fine is a Price” replication to the tax compliance context, discussing these same features for our tax study. In Section 4 we use regression analysis to investigate the sensitivity of our results to individual characteristics. Section 5 concludes.

Section snippets

Replication of the original study: robustness to alternate empirical strategies

Our first study replicates Gneezy & Rustichini’s (G&Rs) original research as faithfully as possible, while translating it to an experimental survey.11

Replication: extending the “fine is a price” mechanism to the tax context

In line with emerging best practices, we use a second study to test the “fine is a price” effect.53 We take our cue from the original article, in which Gneezy & Rustichini suggest that “the announcement of a government that tax evasions are going to be more severely pursued” may generate similar perceptual shifts as those experienced by the daycare parents.54

The possibility that fines for tax evasion may have unintended

Sensitivity of results to individual characteristics

In this section, we use regression analysis to check whether our results on the effect of fines are sensitive to any particular individual characteristics of our sample respondents.106 We

Conclusion

The results from our direct “replication” in the daycare survey do not produce similar results to the daycare field experiment. Instead, we generally obtain results that are more consistent with a standard understanding of the way that fines operate – as a negative consequence that should (other things equal) result in less of the target behaviour. Our tax study, which tests for replication effects in a context suggested by Gneezy & Rustichini, also does not reproduce their key results.

References (0)

Cited by (3)

  • Seemingly irrelevant information? The impact of legal team size on third party perceptions

    2022, International Review of Law and Economics
    Citation Excerpt :

    Specifically, we investigate how the number of lawyers representing an accused person influences third parties’ judgments. Following recent scholarship in law and economics (e.g. Ben-Shahar and Chilton, 2016; Puig and Strezhnev, 2017; Olsen et al., 2018; Metcalf et al., 2020) and management (Croson et al., 2007), we conduct an experimental survey to investigate whether third parties form different judgments about people who engage in similar wrongdoings (i.e. violating the same ethical principles and generating identical harms for identical victims) based on the number of lawyers that represent them. In our experimental survey,1 we present subjects with scenarios which are silent on how the number of lawyers representing a person accused of committing tax evasion and fraud offenses is determined.

Thanks to participants at the 2019 PELS Replication Conference at Claremont McKenna College, especially our commentator Paige Skiba, participants at the 2019 SIOE, CLEA and CELS Conferences and to James MacKinnon and Robin Boadway for helpful comments and discussion of this project; thanks to Suzanne Rath and Kelly Zhang for excellent research assistance. Financial support from the Social Sciences & Humanities Research Council of Canada (Queen’s SIG – Explore Grant - Metcalf) is gratefully acknowledged.

View full text