The unprecedented nature of the COVID-19 pandemic has tasked researchers with describing COVID-19 spread, preventing transmission, finding treatment courses, and evaluating the impact on society and public behavior. Due to the rapid influx of scholarly articles processed by journals and a surge in pre-print articles, research on COVID-19 has swiftly been disseminated. In fewer than 5 months, Scopus has already indexed more than 12,000 publications (Haghani and Bliemer, 2020), with many journals still currently overwhelmed with submissions (Brainard, 2020). While the flood of information has provided scientists and the public with necessary information, a wave of retractions in major journals has begun to appear (Retraction Watch, 2020). When research findings are retracted—especially highly cited and broadly disseminated results—scientific credibility is compromised, and members of the public are often left confused. It is therefore incumbent on both authors and journalists to ensure that scientific findings are properly represented in the media.

Introduction

One potential social impact of COVID-19 that quickly garnered worldwide attention is family violence rates (Mlambo-Ngcuka, 2020).Footnote 1 One study found that COVID-19 related social distancing has been associated with an increase in family violence calls for police service (Leslie and Wilson, in press), and there is reason to hypothesize that family violence offenses would increase with policy interventions designed to mitigate COVID-19 transmission, such as stay-at-home orders. Stress, and the response to stressful situations, are risk factors for family violence, including child abuse, domestic violence and intimate partner violence (Capaldi, Knoble, Shortt and Kim, 2012). For example, with the high unemployment rates occurring as a result of the pandemic, financial strain is a common stressor in many households. There are many theoretical reasons to suspect why this might occur, including unemployment and associated financial strain (Komarovsky, 1940; Schneider, Harknett and McLanahan, 2016), as well as increased rates of substance use (Alexander and Ward, 2018; Reingle, Jennings, Connell, Businelle and Chartier, 2014).

Another common stressor in the home is lack of social support. During the COVID-19 pandemic, there was a notable breakdown in social support networks and many states initiated mandatory stay-at-home (also called shelter-in-place) executive orders, thus lengthening the time spent at home for families (SafeGraph, 2020). Childcare facilities and schools were almost universally closed, employees were laid off because of declining business, and other employees were asked to work from home in an attempt to contain the spread of the virus. Coupled with the financial strain described above, it is conceivable that this increased time at home with competing priorities (i.e., home schooling or babysitting a child while working and maintaining other routine household activities) could increase the likelihood of family violence (Herman, 1992). However, data from family violence victims suggests the severity of victimization remained constant for more than half of victims, and 30% reported less severe victimization during COVID-19 (Jetelina, Knell and Molsberry, 2020). Therefore, the effect of COVID-19 in general—and stay-at-home orders specifically—on family violence remains unclear.

The effect of stay-at-home policies on family violence is an important issue that has clear ramifications for adaptive community-based responses to family violence victimization. The most highly cited research on family violence and stay-at-home policies as of August of 2020 was published in the American Journal of Criminal Justice on June 14, 2020 (Piquero et al., 2020). Piquero et al. (2020) examined the impact of stay-at-home orders on family violence offenses in Dallas, Texas. The study itself was problematic for a number of reasons: 1) the design itself was flawed and did not include more than one year of data for comparison; 2) the analytical methodology was non-conventional and appeared to be biased in favor of detecting a significant association; and, 3) the authors’ conclusions in the manuscript are contradictory. Most problematically, inaccurate findings were disseminated to policymakers, other researchers and members of the public on social media (see Appendix). Each of these issues will be discussed below.

Methodological Problems

Failure to Control for Seasonal and Annual Trends in Family Violence

The study ignored seasonal trends in family violence by failing to account for non-linear or seasonal effects, which are well established in the scientific literature on violence, including family violence (Anderson, Anderson, Dorr, DeNeve and Flanagan, 2000; Koutaniemi and Einiö, 2019). Using a secondary data source from the Dallas Police Department (DPD), seasonal trends can easily be analyzed if robust data cleaning techniques and appropriate statistical analyses are conducted.

The data presented in Piquero et al. (2020) included less than one year of data, which does not allow for comparisons to prior years or spikes in reporting that naturally occur over time. To illustrate the seasonal trends that exist in the data that were used by Piquero et al. (2020), we present descriptive data from the same police department studied in Piquero et al. (2020), which provided us with the daily number of family violence incidents from January 1–May 12, 2020, which we aggregated weekly for presentation purposes. We also present three full years of family violence incident data from DPD in an attempt to visualize the effects of the stay at home order in light of cyclical trends in family violence. Figure displays the average number of weekly family violence incidents from week 1 (January 1–7) to week 52 (December 24–31) of each year. These data suggest that 2020 trends were largely similar to those observed in 2019 and 2018, with slightly fewer incidents reported in January 2020 compared to prior years. Clear seasonal trends are evident, and 2020 trends appear to mirror the trends that were documented in prior years. An increase in family violence incidents was observed between April 1–15 (weeks 14–16) each year.

The time between April 1 and April 15 does not coincide with the implementation of the stay at home executive order (March 23). Our findings of clear seasonal effects are consistent with the scientific literature on aggression generally and family violence specifically (Anderson et al., 2000; Koutaniemi & Einiö, 2019; Rotton and Cohn, 2000; Rotton and Cohn, 2001). This is particularly true given the lower rates of family violence observed in January 2020, as any increase in the incidence rate could reflect an increase in family violence severity and therefore, reporting, rather than an increase in family violence incidence.

On a surface level, our descriptive data demonstrated that the effects of family violence noted to occur in a temporally proximal manner with Dallas County’s stay-at-home order were likely seasonal trends that occurred at roughly the same time each year. Given the well documented cyclical trends in family violence over time, any future attempts to suggest that stay-at-home orders cause an increase in family violence should use multiple years of data and model seasonal trends (Leslie & Wilson, in press). Failing to control for this cyclical nature of the data will likely yield biased estimates in an interrupted time series model (Hylleberg, 1992).

Inadequate Statistical Power to Detect a 12.5% Increase in Family Violence

The study by Piquero et al. (2020) compared family violence rates during the 83 days before the stay at home order was enacted (Code Section 418.10, 2020) to rates for just more than one month (n = 35 days) after the stay-at-home order using secondary data from the Dallas Police Department (DPD). Epidemiological research suggests that even the most robust statistical analyses used to assess the outcomes of public policy changes are insufficiently powered to detect effect sizes of −/+15% (Hawley, Ali, Berencsi, Judge and Prieto-Alhambra, 2019). According to Hawley et al. (2019), thousands of time points would be necessary to detect an effect size of 15%, indicating that the effect sizes in Piquero et al. (2020) were unlikely to ever be statistically significant.

Misrepresentation of Findings to the Media

This study was widely featured on multiple news outlets months prior to scientific publication or even acceptance in a peer-reviewed journal (see Appendix; The Crime Report Staff, 2020). The official university news release (Cutler, 2020) and multiple news outlets quoted a study author as reporting a 12.5% increase in family violence incidents as a result of the stay-at-home orders in Dallas County. This increase (12.5%), nor any other increase of a similar magnitude, was presented in the study.

However, upon closer inspection of the manuscript, none of the associations between the stay at home order and family violence were statistically significant in the first place. The effect size for the time before the stay-at-home order was 1.4%, and the decline thereafter was 12%--neither effect size was significant in the interrupted time series analysis (the beta coefficients were .36 and − 2.49, respectively). In the manuscript, the authors conclude that “the implementation of the stay-at-home order is not associated with a statistically significant increase in domestic violence incidents, and there is not enough evidence to suggest an upward trend in domestic violence incidents throughout the month after the stay-at-home order went into effect” (Piquero et al., 2020, p. 11). This is directly contradictory with the news coverage of the manuscript ten days prior to the manuscript’s acceptance (May 19; Jaramillo, 2020a, 2020b).

In the same manuscript, the authors note that “some of that short-term spike seems to be associated with what appears to be an upward trend of domestic violence crimes that was already occurring prior to the stay-at-home order [emphasis added].” Contrary to this statement, an author of the manuscript was quoted as saying, “I think [the increase in family violence is] strongly associated with the stay at home order” (Fink, 2020). Given the data presented in the manuscript, these contradictory claims are scientifically unjustified, and when widely distributed to news outlets, have potential to inappropriately impact physical isolation policies and potentially, cause harm to the attempts to mitigate COVID-19 spread. In one of the news articles (Fink, 2020), an author also noted:

[The 12.5% increase is] a really strong number because there are not many other things going on that exact date that could have caused such a spike. When you are with someone for a long period of time without much of a way to break, people are going to feel strain. They’re going to feel rage. They’re going to feel a lot of negative emotions and they’re going to lash out unfortunately at the people around them (para. 10).

For this premise to be true, family violence offenses would need to increase during COVID-19, which, based on the author’s data, did not occur. Finally, inaccurate study results, when widely distributed to news outlets, have the potential to impact stay-at-home policies and potentially, cause harm. Stay-at-home orders have been associated with a 60% reduction in COVID-19 cases three weeks after their implementation, with rates increasing each week following their enactment (Fowler, Hill, Obradovich and Levin, 2020). The widespread dissemination of inaccurate research findings, like those described above, have important implications for policy and the virus mitigation efforts, which might urge policymakers to terminate stay-at-home orders in an effort to reduce family violence and other social risk factors. Changes may ultimately result in more COVID-related deaths as stay-at-home orders are prematurely and inappropriately lifted to prevent purported injuries in the home. Therefore, the widespread propagation of these claims in the absence of scientific evidence of an increase has great potential to cause harm.

Conclusions and Summary

It is possible that elevated rates of family violence will occur, although they may be delayed. Informed by the Centers for Disease Control and Prevention’s continuum of pandemic phases models, as well as SAMHSA’s research on coping after traumatic events (Centers for Disease Control and Prevention, 2018; Substance Abuse and Mental Health Services Administration, 2015), projections conducted by the Washington University Department of Health predicted that behaviors indicative of “acting out” (e.g., illicit behavior) are likely to occur between three and six months after the initial outbreak (Washington State Department of Health, 2020). This could be partially responsible for the violence and looting that occurred during protests in response to police violence in June of 2020. We might also expect to see elevations in family violence as the year progresses. The Washington State report also noted that a second wave of illness is expected to generate “large-scale social and economic disruption” (Washington State Department of Health, 2020). This indicates the need to prepare for a widespread surge in family violence if an elevation in family violence rates 3- to 6-months after the initial outbreak are detected.

The goal of this commentary is to prompt a conversation about the potential implications of widely disseminating inaccurate findings from methodologically fallible studies related to COVID-19, with a specific focus on the recent publicity associated with Piquero et al. (2020). The statistical modeling in Piquero et al. (2020) did not model the well-established seasonal trends in family violence (Anderson et al., 2000; Koutaniemi & Einiö, 2019) which are easily identifiable with visual inspection of the data (a multi-year version of the dataset used in Piquero et al., 2020 is provided in Fig. 1). Furthermore, given the discrepancies between the authors’ reports to multiple media outlets and study findings, the authors did not accurately disseminate their results.

Fig. 1
figure 1

Weekly incidents of family violence in Dallas County, 2018-May 2020

Full size image

Given the crucial role that social science can play in informing public policy, it is imperative that scholars work diligently and carefully when disseminating their science. As scholars, we need to ensure that we are using the most robust methods possible to address questions related to important and timely research questions—such as those questions posed by Piquero et al. (2020). Additionally, we must be certain that we understand and honestly report these findings to the media. As scholars we have a professional obligation to ensure not only that we are not the source of this misinformation, but that we actively and publicly correct the misinformation when we become aware of it. As of late August 2020, the authors of Piquero et al. (2020) have only continued to perpetuate the notion that stay-at-home orders increase family violence. Therefore, this continued reiteration of their findings may continue to impact policy to retract stay-at-home orders at the risk of increasing rates of illness and death from COVID-19.

In conclusion, the authors of this commentary respectfully request that scientists publishing policy-relevant findings engage in ethical media engagement practices as well. Specifically, we would suggest that authors of any scientific publication delay media engagement until the peer review process is complete, and this includes dissemination of pre-print manuscripts which have widely increased during COVID-19 (Brainard, 2020). We would also suggest that journalists comply with media ethics guidelines (Society of Professional Journalists, 2014), including to “verify information before releasing it”, “provide access to source information” (the manuscript by Piquero et al., 2020 was not linked in any news article because it had not yet been accepted for publication), and “avoid undercover or surreptitious methods of collecting information”. In this case, there was a clear failure of multiple reporters to verify the claims stated by Piquero et al. (2020), which is a violation of journalistic ethical standards. The authors of this commentary urge the scientific community to engage in a discussion related to the responsible dissemination of research findings, especially those topics that have life and death policy implications.