Insights into auditor public oversight boards: Whether, how, and why they “work”

Highlights

• We survey 170 inspectors from public oversight boards (POBs) in 20 countries to understand whether, why, and how auditors respond to POB oversight.

• Inspectors perceive inspections to have broad effects on several aspects of auditing, ranging from documentation to the removal of partners.

• Some inspectors perceive that auditors place greater weight on keeping fees low than on increasing audit quality.

• Auditors are perceived to respond to POB feedback to avoid criticisms in publicly disclosed inspection reports and enforcement actions.

• In addition, POBs are perceived to be effective at improving audit quality due to their authority and culture for detecting auditing deficiencies.

Abstract

We survey 170 inspectors, representing 27% of the inspection staff, from auditor public oversight boards (POBs) in 20 countries to understand whether, how, and why auditors respond to POB oversight. We find that a large majority of inspectors believe that auditors frequently respond to their feedback by changing audit procedures and quality control systems. Inspectors perceive inspections to have broad effects on several aspects of auditing, ranging from documentation to the removal of partners. Some inspectors perceive that auditors place greater weight on keeping fees low than on increasing audit quality. Inspectors also believe that auditors on occasion ‘fix’ closed audit files before an inspection, and in rare instances obtain confidential information about upcoming inspections. Inspectors think that the primary reasons why auditors respond to POB feedback are (1) public disclosure, (2) enforcement capabilities, (3) POBs being perceived as authoritative, and (4) POBs having a culture for detecting auditing deficiencies.

Introduction

Over the past two decades, several countries have changed their approach to oversee the auditing profession, moving from a system of self-regulation to one where an independent regulator oversees auditors. Specifically, many countries have established public oversight boards (POBs). The creation of POBs was, in part, a response to the accounting scandals of the early 2000s based on the presumption that regulatory oversight would be a more effective approach to oversee auditors and address the conflicts of interest between auditors, their clients, and investors than the self-regulation approach (Coffee, 2001; Coates and Srinivasan, 2014).

From a theoretical standpoint, there are several limitations of a regulatory solution (i.e., POB) and neither approach to auditor oversight, public oversight or self-regulation, dominates the other (Minnis and Shroff, 2017). For example, public regulators are susceptible to “capture” by special interest groups (Stigler, 1971), such as the big-four audit firms in our context. Further, depending on the regulator's source of funding, it could come under political pressure that undermines its effectiveness (Peltzman et al., 1989). While POBs are designed to be independent of the accounting profession to address the above issues, it is debatable as to whether this goal is achieved. In addition, critics contend that POB inspectors do not have the in-depth expertise that practicing auditors have and argue that the POB approach has sacrificed expertise for independence (Palmrose, 2006; Glover et al., 2009). Further, regulatory organizations typically cannot compensate their staff as well as the private sector, potentially affecting their ability to attract and retain talent or incentivize them to advance the public interest. Regulators are also subject to “revolving doors,” (i.e., working in the public section with the hopes of attaining a better job in the private sector) which can limit their effectiveness (Grossman and Helpman, 2001).

Several recent studies examine whether changes in auditor regulatory oversight affect proxies for audit quality, primarily focusing on the Public Company Accounting Oversight Board (PCAOB) in the U.S. (Carson et al. (2021) is an exception because they examine multiple POBs). The weight of the empirical evidence suggests that POB (again, primarily PCAOB) inspections improve audit quality and confer economic benefits to clients of inspected auditors.¹ However, given the theoretical limitations of a public regulatory approach to oversee auditors and because there are strong critics of the POB model, it is important to understand how and why POB inspections are effective. Our study endeavors to contribute data to this quest.

Empirically testing how POB inspections change auditor behavior using the archival approach would require granular data on the audit work papers for several audit engagements before and after the inception of POB oversight. Similarly, testing why POBs are effective at improving audit quality using the archival approach would require data on the internal workings of POBs, such as employee incentive plans, promotion criteria, inspection approaches, governance structures, etc. for several POBs, as well as similar data on the alternative oversight approach that the POB replaces. Given unavailability of such data to conduct an archival study, we conducted a large-scale survey of POB inspectors around the world to begin to answer these questions.

Survey data enable us to gain insights from firsthand accounts of POB inspectors into the factors that motivate them to improve audit quality and understand the mechanisms POBs have in place to address typical shortcomings of regulatory organizations. A typical inspection involves an examination of the audit work papers for several audit engagements and an evaluation of the auditors’ quality control systems. Further, the typical inspection process entails face-to-face interactions between inspectors and auditors during the inspection fieldwork. As a result, POB inspectors directly observe the effect of their inspections on changes in auditing procedures and practices over time for a broad cross-section of audit firms and audit engagements, making survey data uniquely suitable to investigate our research questions.

Despite the unique data obtained via a survey, we recognize that there are important limitations to survey methodology. Besides the typical limitations of a survey (e.g., do respondents understand the questions as intended, selection concerns regarding who completes the survey), an important concern specific to our research question is that we ask inspectors about how and why they perceive inspections to affect auditor behavior. Given that inspectors are choosing to work at a POB, their views could be favorably biased about the effectiveness of POBs. We discuss our efforts to mitigate the effects of such a bias on our inferences later in the paper.

We begin our analyses by examining whether POB inspectors perceive that inspections affect auditor behavior (before examining inspector perceptions on how and why POBs affect auditor behavior). Since the vast majority of prior studies that examine the effect of POB inspections focus on the PCAOB (again, Carson et al. (2021) is an exception), we utilize our survey to expand this examination to POBs in 20 countries. In addition, most prior studies examine changes in audit outcomes after a PCAOB inspection and infer that the way in which audits are performed must have changed. A unique benefit of the survey approach is that we can obtain data about specific audit processes that change in response to inspections. As such, we use survey data to gain insights not only about whether non-U.S. POBs matter, but also which specific aspects of the audit process change (as perceived by inspectors).

Data from the survey responses of 170 audit inspectors employed by POBs in 20 countries reveal that 78% of the inspectors perceive that auditors “frequently” or “very frequently” adjust general audit procedures (and 69% respond similarly about quality control systems) as a result of the inspections. 46% of inspectors observe that auditors “frequently” or “very frequently” change culture or the tone at the top in response to inspection feedback. Because most of the motivation for our analyses stems from prior research on PCAOB inspections, we separately examine survey responses for all questions for inspectors from four POBs – Netherlands, Norway, the U.K., and the U.S. – that are most representative of the type of oversight provided by the PCAOB.² We highlight major differences in the perceptions of inspectors from these four countries when such differences arise and are statistically significant. In terms of the question of whether inspections change auditor behavior, we find that inspectors from the Netherlands, Norway, the U.K., and the U.S. (henceforth, NNUU inspectors) report observing more frequent changes in general audit procedures for inspected clients and quality control systems than the inspectors from other POBs.

We then examine inspector perceptions of how auditors respond to inspections. Survey responses reveal that auditors “frequently” or “very frequently” respond to inspection feedback by increasing documentation (86%), conducting firm-wide training (83%), increasing the amount of audit/testing effort (64%), increasing scrutiny of management estimates (64%) and changing the audit-quality review process (62%).³ The survey responses (i.e., inspector perceptions) also suggest that audit firms on occasion respond to POB inspections by removing lower quality partners from the audit engagements of public companies (39%), changing controls over auditor independence (38%), and changing compensation policies for engagement partners (30%). That more than 30% of the inspectors indicate that changes to partner compensation and the removal of lower quality partners occur frequently or very frequently suggest that inspections have far-reaching effects on how audits are conducted. To our knowledge, these are the first data gathered that describe the many changes audit firms make to audit processes in response to POB inspections.

To validate our survey results, we examine whether inspector responses to the above question are associated with changes in unsigned discretionary accruals, our proxy for audit quality. Following Carson et al. (2021), we regress audit quality (measured at the company-year level) on both (1) the determinants of audit quality from the archival literature and (2) inspector survey responses, aggregated at the POB-level. We find that POB inspections are negatively associated with unsigned discretionary accruals. Importantly, we find that the effect of POB inspections on unsigned discretionary accruals is stronger when inspectors perceive that auditors respond to inspector feedback by (1) increasing documentation, (2) conducting firm-wide training, (3) increasing scrutiny of estimates, (4) increasing testing, and (5) removing lower quality partners from public company audit engagements. Tying cross-sectional variation in an archival proxy for audit quality to inspector perceptions regarding how auditors respond to inspections, mitigates the concern that inspector perceptions collected via our survey are disconnected from reality. In addition, these results provide descriptive insights into which specific audit input changes lead to improvements in audit quality.

Another way in which auditors might respond to inspections, is by gaming the inspection system. Thus, we ask inspectors how often they think that auditors (1) “fix” archived audit files to prepare them for inspections and (2) inappropriately obtain confidential information about upcoming inspections. We find that 8% of the survey respondents indicate that auditors “Never” tamper with closed/archived files before inspections, while 48% respond by indicating such behavior is perceived to be rare. When asked whether auditors inappropriately obtain confidential information about upcoming inspections, 33% responded “Never,” and 48% responded such behavior is rare. Correlating the responses to these questions to archival data, we find that POB inspections have a weaker effect on audit quality when inspectors perceive that audit firms obtain confidential information about upcoming inspections, consistent with our expectations.

We then examine inspector perceptions about why POB oversight affects audit quality/practice. 80% of all inspectors in our sample, and 90% of NNUU inspectors, “agree” or “strongly agree” that auditors respond to inspections because they do not want the inspection report to contain criticisms of their audit procedures. 74% of all inspectors (81% of NNUU inspectors) “agree” or “strongly agree” that auditors respond to their feedback to reduce the risk of enforcement actions and fines.⁴ Our finding that the disclosure of inspection reports is perceived to be an important tool to change auditor behavior is relevant for policy makers because the POBs in most countries are prohibited from publicly disclosing inspection findings at the auditor-level.

We also ask the converse of the question above—i.e., in circumstances where auditors do not respond to inspection feedback, why do they not respond? A majority of inspectors (57% in the full sample and 68% of NNUU inspectors) “agree” or “strongly agree” that auditors do not respond to inspector feedback when they believe the mistake identified is a one-off error. We also find considerable support for the idea that inspectors perceive that auditors place greater weight on keeping fees low than on increasing audit quality, which likely reflects the preferences of audit firms' clients.^5,6 Since audits possess characteristics of a credence good (Causholli and Knechel, 2012), where financial statement users do not observe audit quality, the demand for audits could become a function of audit fees. Thus, our evidence is consistent with POBs addressing a market failure created by investors’ inability to observe audit quality.

We also ask inspectors to compare the strengths and weaknesses of POB inspections to peer reviews. We find that the majority of respondents think that POB inspectors have greater authority (enforcement options) than peer-reviewers and that the culture at POBs is more conducive to detecting auditing deficiencies.

We next examine whether inspectors are provided economic incentives to conduct high-quality inspections and detect audit failures during inspections. The data reveal that inspectors have weak performance related incentives (in terms of promotion prospects, private sector opportunities, etc.). We find little evidence that inspectors seek employment at POBs with a desire to return to the private sector for higher pay (i.e., a ‘revolving door’); but rather, the majority of the inspectors working at a POB say they do so for greater work-life balance combined with their intrinsic interest in auditing. When correlating survey responses with discretionary accruals, we find that POBs with formal inspection evaluation programs are more likely to see audit quality improvements following the commencement of their inspections.

Our paper contributes to the literature on regulation and auditing in two ways. Prior research almost entirely focuses on examining outcomes from the audit process and inferring that inputs into the audit process must have changed following the commencement of POB inspections. For example, some studies examine the consequences of the shift from self-regulation to public oversight on the auditor market share (Lennox and Pittman, 2010; Aobdia and Shroff, 2017), auditor exits from the audit market (DeFond and Lennox, 2011), audit quality (Lamoreaux, 2016; DeFond and Lennox, 2017; Fung et al., 2017; Krishnan et al., 2017; Aobdia, 2018; Ege et al., 2019; Gipper et al., 2020), and external financing and investment (Shroff, 2020; Aobdia et al., 2021). Other studies examine the credibility and information content of peer-review reports and compare it to those prepared by POB inspectors (Hilary and Lennox, 2005; Anantharaman, 2012). We contribute to the literature using unique survey data to gather insights into inspector perceptions of how auditors respond and why inspectors perceive POBs to be effective at changing auditor behavior. Public oversight is still a relatively new approach to regulate audit firms, and many countries, including the U.S., periodically evaluate the organizational structure and operations of POBs, implementing changes as they receive feedback. The findings from our paper provide insight into inspectors’ perceptions from different POBs regarding the factors that make POBs effective. These insights could help regulators learn from the experiences of many POBs and inform them as they consider re-organizing the operations and structure of their own POB.⁷

Our paper is also related to prior studies where authors surveyed and interviewed practicing auditors and found that U.S. auditors and audit firms are very critical of PCAOB inspections, questioning the value of such oversight. For example, Glover et al. (2009) criticize the PCAOB's incentive system, organizational composition and structure, calling the organization “inefficient and dysfunctional.” Westermann et al. (2019) survey and interview auditors of varying ranks and conclude that PCAOB inspections have led audit firms to resort to “impression management strategies and ‘functionally stupid’ work practices…”⁸ Our survey of POB inspectors provides another point of view that can help us better understand the effect of POB oversight on the auditing profession. POB inspectors and auditors are perhaps the two most informed groups about the effects of POB oversight. Yet, the perceptions of these agents regarding the value of POB oversight and their inspection experiences differ likely because (1) it is one group's job to identify mistakes in the other group's work and (2) inspectors and auditors self-select into these roles and thus are inclined to believe that what they are doing is valuable/correct. An important advantage of collecting the perceptions of POB inspectors is that their perceptions are based on the experiences gained from inspecting a broad cross-section of audit firms and audit engagements. In contrast, most individual auditors are likely to have had relatively idiosyncratic experiences based on when their engagement was selected for a POB inspection, and their perception of the inspection process is likely to be influenced by the outcome of that one inspection (or those few inspections).⁹

Because there are such disparate beliefs among academics and practitioners regarding the efficacy of POBs, some of the data from our survey will likely be surprising to some and perhaps controversial in certain regards. We hope the data provide insights that further our understanding of POBs and their effects. At the same time, as discussed earlier, and as we discuss more below, we caveat that there are important limitations to survey research—possible selection issues, misunderstanding of questions, potential incentives to lie, inspectors being potentially biased favorably toward positive outcomes of the POBs, etc. While there is no way to eliminate such concerns, we take several precautions, discussed below, to mitigate such concerns.