Is a fine still a price? Replication as robustness in empirical legal studies☆
Introduction
Uri Gneezy and Aldo Rustichini published “A Fine is a Price” in the Journal of Legal Studies in 2000.1 The authors used a randomized field experiment at Israeli daycares to test whether “when negative consequences are imposed on a behaviour, they will produce a reduction of that particular response.” Instead of validating this standard assumption in psychology, economics, and legal studies, the authors found that introducing a fine increased late behaviour in their experiment.
Gneezy & Rustichini offered two possible explanations – both of which involved the fine transforming parents’ perceptions of the context. The first explanation was that the fine provided information about the consequences of coming late that had been unspecified in the contract between parents and the daycare. While parents may have previously feared severe consequences from coming late, introduction of the fine made it clear this was not a threat.
Gneezy and Rustichini offered another powerful explanation: before the fine existed, a social norm restrained parents from coming late. Once the fine was introduced, it transformed the parents’ perceptions of the extra care into a commodity. They were now free to consume it at the “price” of the fine, without the guilt or shame felt when transgressing the social norm. Parents’ internal, moral motivations were “crowded out” by the market signal sent by the fine. Gneezy & Rustichini’s experiment has been widely accepted as support for this latter, intuitively appealing theory. The paper has been cited more than 2000 times including in popular non-fiction for general audiences.2
Gneezy and Rustichini’s social norms explanation fit well with emerging developments in economic theory that expanded standard approaches to individual preferences. Theorists developed models for individual preferences that reflected altruism and other-regarding behaviour, drawing on empirical support from standard economic games.3 Another strand of emerging theory developed models of preferences dependent on reference points, providing a related theory of how a shift in behaviour might occur.4 Scholars also provided models and empirical evidence consistent with individuals behaving differently in contexts perceived as social as opposed to market exchanges.5
Despite these developments, questions remain about the extent to which social norms influence choices in economic environments and whether fines “crowd out” the effects of norms, as Gneezy and Rustichini suggested.6 Most relevant for our project, there has been some criticism of Gneezy and Ristichini’s original field study7 and the authors themselves suggested potential benefits from replication, even undertaking their own follow-up study.8
Given the significant influence of the original paper and the importance of the prospective behavioral insight it uncovers, we believe it is a good candidate for a form of replication study. However, in the case of field studies like Gneezy and Rustichini’s original, faithfully reproducing the full set of underlying conditions embedded in the social and temporal context for the field experiment is challenging. This would make it difficult to view any inconsistent result as a true failure to replicate. For empirical work that is inherently contingent on complex background factors such as legal institutions, culture and political conditions, we suggest that a “many studies”9 approach as a way to test robustness of the underlying mechanism may be more appropriate than strict replication. Empirical work and results like this are perhaps best conceptualized as an ongoing project that inherently involves the collaboration of a community of researchers. The creation of a body of related work will ground confidence in the insights and an understanding of their limits.
With this approach in mind, we conduct a number of related studies to explore the robustness of Gneezy & Rustichini’s original results. We transform the original daycare field setting into a vignette-based experimental survey administered on Amazon’s Mechanical Turk (MTurk) platform. This approach allows us to control aspects of the experimental design that are difficult to replicate in the field, and to directly investigate the alternative explanations Gneezy and Rustichini offered for their original results. We also test for evidence of the underlying mechanisms they identify in a tax compliance setting – an alternate context where Gneezy and Rusitchini suggested the “fine is a price” effect could appear. Survey-based research in empirical legal studies has grown, particularly as accessibility through online platforms such as MTurk has expanded.10 We seek to replicate Gneezy & Rustichini’s famous result with this increasingly common approach. While our study is thus not a direct opportunity to confirm or refute the original field trial findings, we hope to provide evidence relevant to their generality and contribute to the emerging literature employing MTurk in empirical legal studies, psychology and economics.
Our survey results do not replicate the original findings. In our daycare and tax studies, the introduction of fines causes a reduction in anticipated non-compliant behaviour by our survey respondents. Moreover, subjects anticipate that other respondents will be similarly affected. Fines do not cause subjects to adjust their concerns about an incomplete contract in a manner that is consistent with Gneezy & Rustichini’s theory. They also show very little evidence of fines crowding out any social norms. Moreover, the effect of fines on outcome behaviours and the reasons behind the outcomes of our respondents is transitory. Once the fines are removed, they return to their baseline behaviours. These survey results are consistent with our intuitive judgments (and standard rational-choice expectations) that fines deter.
The remainder of the paper is organized as follows. In Section 2 we introduce our primary replication of Gneezy and Rustichini’s daycare study. Section 2 includes discussion of our experimental structure and survey instrument, survey implementation and data collection, and results. In Section 3 we extend the “Fine is a Price” replication to the tax compliance context, discussing these same features for our tax study. In Section 4 we use regression analysis to investigate the sensitivity of our results to individual characteristics. Section 5 concludes.
Section snippets
Replication of the original study: robustness to alternate empirical strategies
Our first study replicates Gneezy & Rustichini’s (G&Rs) original research as faithfully as possible, while translating it to an experimental survey.11
Replication: extending the “fine is a price” mechanism to the tax context
In line with emerging best practices, we use a second study to test the “fine is a price” effect.53 We take our cue from the original article, in which Gneezy & Rustichini suggest that “the announcement of a government that tax evasions are going to be more severely pursued” may generate similar perceptual shifts as those experienced by the daycare parents.54
The possibility that fines for tax evasion may have unintended
Sensitivity of results to individual characteristics
In this section, we use regression analysis to check whether our results on the effect of fines are sensitive to any particular individual characteristics of our sample respondents.106 We
Conclusion
The results from our direct “replication” in the daycare survey do not produce similar results to the daycare field experiment. Instead, we generally obtain results that are more consistent with a standard understanding of the way that fines operate – as a negative consequence that should (other things equal) result in less of the target behaviour. Our tax study, which tests for replication effects in a context suggested by Gneezy & Rustichini, also does not reproduce their key results.
References (0)
Cited by (3)
Seemingly irrelevant information? The impact of legal team size on third party perceptions
2022, International Review of Law and EconomicsCitation Excerpt :Specifically, we investigate how the number of lawyers representing an accused person influences third parties’ judgments. Following recent scholarship in law and economics (e.g. Ben-Shahar and Chilton, 2016; Puig and Strezhnev, 2017; Olsen et al., 2018; Metcalf et al., 2020) and management (Croson et al., 2007), we conduct an experimental survey to investigate whether third parties form different judgments about people who engage in similar wrongdoings (i.e. violating the same ethical principles and generating identical harms for identical victims) based on the number of lawyers that represent them. In our experimental survey,1 we present subjects with scenarios which are silent on how the number of lawyers representing a person accused of committing tax evasion and fraud offenses is determined.
Review of behavioral economics models of the altruistic crowding-out effect from monetary incentives
2023, Journal of Economic Surveys
- ☆
Thanks to participants at the 2019 PELS Replication Conference at Claremont McKenna College, especially our commentator Paige Skiba, participants at the 2019 SIOE, CLEA and CELS Conferences and to James MacKinnon and Robin Boadway for helpful comments and discussion of this project; thanks to Suzanne Rath and Kelly Zhang for excellent research assistance. Financial support from the Social Sciences & Humanities Research Council of Canada (Queen’s SIG – Explore Grant - Metcalf) is gratefully acknowledged.