Design and evaluation of an advanced continuous data level auditing system: A three-layer structure

https://doi.org/10.1016/j.accinf.2021.100524Get rights and content

Highlights

  • We propose a CA system with a three-layer structure to identify various types of suspicion transactions and ultimately provide assurance on financial statement accounts.

  • This architecture categorizes the suspicion transactions into three categories: (1) unusual transactions are labeled as “non-routine errors,” (2) transactions that violate internal control are labeled as “exceptions”, and (3) transactions that statistically deviate from the organization’s standard business behaviors are labeled as “anomalies”.

  • To examine whether the proposed design enhances the effectiveness of a continuous auditing system in identifying financial irregularities, this study empirically tests the proposed models by using real-world journal entry transaction data from a construction company.

  • The results show that the proposed design outperforms traditional models, which imply that a three layer structure could improve and add value to the continuous data level auditing system.

Abstract

Audit efficiency and effectiveness can be significantly affected by data aggregation during audit procedures. Previous studies highlight that an appropriate level of data aggregation is needed because a continuous auditing (CA) system often generates numerous alarms. To respond to this issue, this study proposes a CA system with a three-layer structure. In the first layer of the proposed system, all journal entry level transactions are classified and aggregated using defined rules; any transactions that deviate from these rules are identified as unusual transactions. The second layer detects the observations that violate controls. Analytical monitoring models are developed in the final layer to identify observations that statistically deviate from an organization’s typical business behaviors. To examine whether the proposed three-layer CA system enhances the effectiveness of a CA system in identifying financial irregularities, this study empirically tests the proposed models using real-world journal entry data from a construction company. The results indicate that the proposed framework enhances audit effectiveness and efficiency.

Introduction

A continuous auditing (CA) system is used “to provide assurance virtually simultaneous with, or within a short period after, the occurrence of events underlying the subject matter” (CICA/AICPA). A CA system enables auditors to examine more disaggregated data in near real-time, potentially increasing audit effectiveness (Alles et al., 2006) but tends to generate too many alarms related to suspicious transactions (Jans and Hosseinpour, 2019, Perols and Murthy, 2012). Audit efficiency is significantly influenced by the number of suspicious activities discovered (Li et al., 2016).1 Therefore, it is necessary to identify an appropriate level of data aggregation for CA systems (Kogan et al., 2014).

The use of aggregated data in a CA system presents advantages as well as disadvantages. Aggregated data provide a small number of observations for analysis, and thus easier to use in creating a stable model than disaggregated data (Cogger, 1981). However, the use of aggregated data can lower the possibility of detecting errors at a detailed level. To address this issue, previous studies design CA systems with different structures (Kogan et al., 2014, Perols and Murthy, 2012). Nevertheless, Kogan et al., 2014, Zhang et al., 2015 point out that the data aggregation level needs to be determined based on different scenarios with the consideration of inherent characteristics and the risk level of underlying transactional data. Thus, additional research in this area is needed (Zhang et al., 2015).

In this study, we propose a CA framework that is designed mainly to identify various types of suspicious transactions and ultimately provide assurance regarding financial statement accounts. This architecture separates suspicious transactions into three categories; (1) transactions that are outside the normal course of a firm’s business are labeled “unusual transactions” (AICPA, 2002), (2) transactions that violate internal controls, are labeled “exceptions” (Kogan et al., 2014), and (3) transactions that statistically deviate from an organization’s standard business behaviors are labeled “anomalies” (Kogan et al., 2014). This three-layer structure allows auditors to conduct further appropriate investigations because each layer provides different insights regarding the possible issues surrounding a firm’s business transaction process. The proposed CA system also enables auditors to examine transactions without a large volume of alarms, while maintaining a reasonable number of observations for analytical statistical power.

The proposed architecture is built on an initial set of rules based on accounting principles and standard business activities related to respective transactions. The rules are based on the relevant ledger, accounts, and the credit or debit side (increase or decrease) of those accounts. For example, one of the rules concerns transactions in the sales ledger that involve a debit to accounts receivable and a credit to revenue accounts. In the first step, transactions are classified into relevant groups by using the predefined rules, which enable auditors to identify unusual transactions as indicators of potential fraud risk (AICPA, 2002). The second step filters exceptions based on deterministic rules (identifying internal control violations) to identify unreconciled differences in relevant accounts. Only observations that do not violate any control can advance to the final step: anomaly detection. Anomalies are detected by implementing the analytical procedure component of the continuous data level auditing system. Expected behaviors of a given group are estimated based on the relationships among different observations and external economic indicators. When an observation deviates significantly from the expected behavior, it is identified as an anomaly. For example, the expected increase in revenue from sales can be estimated by a regression equation on the prior period’s revenue and the consumer price index, and if an observation shows an increase that differs significantly from the prediction, it is identified as an anomaly.

To test whether the proposed system outperforms the models suggested in previous research, this study utilizes the transactional data of a construction firm from 2010 to 2015 and empirically evaluates the performance of the proposed continuous data level auditing system. The performance of our system is evaluated based on its ability to identify errors. For comparative purposes, we examine how well the system captures errors correctly by using three different data formats: (1) financial statement data (most aggregated data), (2) journal entry data (most disaggregated data), and (3) journal entry data aggregated in terms of the predefined rules (semi-aggregated data, which is the data format used in the proposed CA system).

The empirical results show that the proposed CA system developed from semi-aggregated journal entry data identifies errors more effectively than a CA system developed using financial statement data or journal entry data. As expected, models that use aggregated financial statement data tend to be more efficient (i.e., fewer false positives) but less effective (i.e., more false negatives) in identifying errors than disaggregated journal entry data. However, some models that use journal entry data do not identify errors more effectively than those that use financial statement data since journal entry data fluctuate more than financial statement data. Moreover, by utilizing the proposed system, journal entry data are more likely to enhance the effectiveness of CA compared to benchmark CA systems that use financial statement account data and apply models without rules to identify anomalies. This tendency indicates that the proposed framework can improve the performance of existing CA systems.

This study contributes to the academic literature by proposing a CA system with a three-layer structure. Data aggregation and audit efficiency issues (i.e., how to deal with a large volume of alarms) are addressed by the proposed CA system. Auditors, provided with disaggregated data based on the predefined rules, can now focus on the suspicious transactions identified by the three layers. With the emergence of big data, auditors are faced with large volumes of data from CA systems, which requires additional audit effort. Thus, it is necessary to identify an appropriate level of data aggregation to improve audit efficiency and effectiveness (Zhang et al., 2015).

The remainder of this paper is organized as follows. Section 2 constitutes the motivation behind this study and summarizes prior literature. Section 3 illustrates the design of an advanced continuous data level auditing system, while Section 4 presents empirical results, including robustness tests. Finally, Section 5 presents conclusions and possible future research areas.

Section snippets

Motivation and background literature

CA was pioneered by Groomer and Murthy, 1989, Vasarhelyi and Halper, 1991. Groomer and Murthy (1989) introduce embedded audit modules, and Vasarhelyi and Halper (1991) apply a control and monitoring layer to alarm auditors as to when a transaction violates predefined rules. CA differs from traditional auditing in seven ways, as stated by Chan and Vasarhelyi (2011): (1) continuous or more frequent audits; (2) proactive audit model; (3) automation of audit procedures; (4) evolution of the work

Overview

The objective of the proposed CA system is to provide near-real-time assurance on financial statement accounts. The architecture of the proposed CA system is designed based on procedures implemented by the system equivalent to tests of control and substantive analytical procedures of transactions in traditional auditing (Kogan et al., 2014). Each component deliverers different insights. Through two separate components, three different alarms are individually identified as Table 1 shows.

Data

The data used in this study are provided by one of the largest global accounting firms. The data include journal entry level accounts of daily transactions of the construction company from 2010 to 2015. The original data used for our analysis contains 57,786 daily transactions at the journal entry level. Examining data from a construction company provides unique but limited value in testing the proposed architecture. The construction industry represents a major sector of the economy of the

Conclusions and study limitations

A large volume of alarms in a CA system can reduce audit efficiency. This may lead to a CA system that relies on aggregated data. However, using aggregated data can harm audit effectiveness. Therefore, it is important to identify an appropriate level of data aggregation when building a CA system.

This study establishes how auditors can continuously monitor an organization’s accounting information system using a three-layer structure: (1) data aggregation and transaction verification to identify

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

We appreciate a large accounting firm for giving us access to the financial data and supporting us on this research project. The authors appreciate the helpful comments and suggestions from Lucas A. Hoogduin, Alexander Kogan, Michael Alles and the AIS group at Rutgers Business School. This study is benefited from the helpful suggestions from the 2018 Joint Mid-Year Meeting of the Accounting Information Systems Section and the Strategic and Emerging Technologies Section, especially the

References (38)

  • Y. Chen et al.

    The error detection of structural analytical procedures: A simulation study

    Auditing: J. Pract. Theory

    (1998)
  • Y. Chen et al.

    An analysis of the relative power characteristics of analytical procedures

    Auditing: J. Pract. Theory

    (1999)
  • CICA/AICPA, Research Report: Continuous Auditing. The Canadian Institute of Chartered...
  • M. Codesso et al.

    Continuous Audit Implementation at Cia. Hering SA in Brazil

    J. Emerg. Technol. Account.

    (2020)
  • K.O. Cogger

    A time-series analytic approach to aggregation issues in accounting data

    J. Account. Res.

    (1981)
  • M.M. de Freitas et al.

    Implementation of Continuous Audit on the Brazilian Navy Payroll

    J. Emerg. Technol. Account.

    (2020)
  • R.S. Debreceny et al.

    Embedded audit modules in enterprise resource planning systems: implementation and functionality

    J. Informat. Syst.

    (2005)
  • S.C. Dzeng

    A comparison of analytical procedure expectation models using both aggregate and disaggregate data

    Auditing: J. Pract. Theory

    (1994)
  • S.M. Groomer et al.
  • Cited by (0)

    View full text