Theory building via agent-based modeling in public administration research: vindications and limitations

Generative/emergent

ABM is a methodological tool equated closely with what Epstein (1999) called “generative social science” (Cederman, 2005), although “emergent” is a more popularly used word today. By generative, Epstein (1999, p. 44) meant that ABM is not inductive but deductive in that the method generates computational results-a macro-level structure built up on a given set of assumptions of agents, rules and environments. In the similar epistemological vein, Smith and Conrey (2007, p. 88) emphasized the bridging role of ABM “between the micro level of assumptions regarding individual agent behaviors, interagent interactions, and so forth and the macro level of the overall patterns that result in the agent population.”

Schelling's classical model of spatial segregation (1971) is a good example to demonstrate the nuts and bolts of ABM. The model generates a map of spatial segregation exhibited between two groups of agents. It happens by simply using a threshold rule that agents move to another place on the map when they want some of the neighbors to be proximate to the same group. An actual simulation shows a surprising degree of spatial segregation generated from the simple rule, even when the threshold is low.

There have been impressive arguments endorsing theoretical status of ABM with its methodological competency. Scholars using ABM contend that the model comes out of theoretical review and the model is itself an apparatus of theoretical conception. Some even go further to argue that the model is itself a theory (Carley and Gasser, 1999; Cohen and Cyert, 1965; Harrison et al., 2007; Sun, 2009). A specific agent-based model connotes that an emergent social phenomenon such as spatial segregation (Schelling, 1971) can be sufficiently generated by a couple of simple rules that agents follow. Consequently, as argued by Epstein (1999, p. 42), computational demonstration of macrostructure is taken as a necessary condition for explanation itself. The connotation stimulates us to say that ABM corresponds to a structure of a theory, as well as of a methodology.

Computational

The generative power of ABM is supported by its computational attributes. According to Carley (2002, p. 7257), who contributed to development of ABM and social simulation in general, a computational organization science is a “a neo-information-processing approach … that combines social science, computer science, and network analysis.” This approach supports perceiving groups, organizations or social systems as distributed intelligent systems composed of multiple agents (Weiss, 1999), and their interactions as ancillary patterns of information sharing and learning.

The computational nature of ABM has two pronounced merits. First, simulation results can differ depending on parameter values. Staying this way, ABM can be not simply a different logical reasoning avenue to get to formal deduction, but a good operational tool useful for addressing the stochastic nature of social issues. For example, simulations by the Schelling model yield different shapes of spatial segregation; all the results are a deduction from the initial condition and rule. However, the computational nature of the model renders the simulation results to be more explicit variants or nuanced versions of formal deduction. Second, ABM can demonstrate “surprising” conclusions from the initial assumptions. Spatial segregation may occur in the Schelling model even when agents want to see only one-third of their neighbors proximate to the same group (Schelling, 1971). As such, not all logical consequences of agents' local interactions can be expected from the verbal reasoning that follows the previously set assumptions about the agents. In this way ABM can be coupled with sophisticated computational designs and exhibit a wide range of computational consequences out of theoretical assumptions (Choi and Robertson, 2014b).

Agent

A researcher may first define an agent as the basic element of the model. Since ABM “grows” a group, organization or society through multiple agents' interactions, the definition of key agents is the starting point for maneuvering ABM. An agent is not necessarily a human being equipped with cognitive agency; it can be whatever entity capable of processing information, making decisions and taking actions (e.g. an organization can also be an agent). What is important is not the unit of an agent, but the researcher's theoretical concern-to proceed from what level to grow the society.

An agent possesses two elements: properties and functions (Figure 1). The former includes characteristics of agents such as goals, information, resources and demographics. In the Schelling model, for instance, an agent can be defined as an entity possessing properties such as social preference (in the original model) and income, if necessary. The latter defines the agent's behaviors such as exploring, sharing, updating information and handling resources. In the Schelling model, agents “recognize” part of their neighbors, and “move.” Similarly, one can simulate group polarization during a deliberation process. In this case, the agent is individual group members who possess initial preferences and information, and contact with other members and change preferences. In practice, agents are often designed as boundedly rational, and in such a way to adapt to the outside conditions through learning from local interactions. In actual models, agents usually move (Epstein and Axtell, 1996; Schelling, 1971), search (Maroulis and Wilensky, 2015), vote (Choi and Robertson, 2014b) and make coalitions (Scott et al., 2019), while updating their properties and functions according to feedback (arrows in Figure 1). The concept of “complex adaptive systems” or CAS (Holland, 1995) has often been used to represent the adaptive capabilities of agents.

Rules

The second building block is rules. Rules are typically signified by “IF… THEN…” as denoted by interacting arrows in Figure 1. Rules, first of all, include the actions of agents. For example, an agent would obtain information from another agent based on their homophily; in this case, the agent's decision is affected by its preference, but the action is based strictly on the abstract rule that reflects the preference: “agree if and only if the information is compatible with your preference.” Second, rules include norms and institutions beyond the reach of an agent. For example, although each agent may act to vote for or against an agenda, there can be a group level decision rule – unanimity or majority (Choi and Robertson, 2014b). That being said, rules in ABM are usually embedded in agents or the environment in the form of function of interactions.

More recently, scholars have been at work to combine the concept of Institutional Grammar with ABM (Siddiki et al., 2019, p. 19). In this case, the decisional rules posit institutional statements as “shared linguistic constraint or opportunity that prescribes, permits, or advises actions or outcomes for actors” (Crawford and Ostrom, 1995, p. 583). In the same vein, the rules are “categorized and dissected in accordance with the appropriate linguistic syntax” (Siddiki et al., 2019, p. 5). These are illustrative of the pronounced attempt to magnify the central role assumed by rules in theory building.

How a researcher defines rules should be determined by relevant theoretical propositions or hypotheses, whether empirical or logical. In the case above, a preference based on homophily and resulting behavior can be designed according to what has been found in group research (De Drew et al., 2008). Then, why a certain macro-level structure has emerged becomes a query to be explained by referring to those rules. Therefore, it would be asserted that relevant theoretical assumptions can and should be included in a model as rules.

Environment

The third building block of an agent-based model is environment. A design of environment is related to “controlled” macro-level variables or conditions in which agents play. Schelling's (1971) spatial segregation model employed a two-dimensional grid space as environment. Some study in the organizational population ecology designed more abstract environment to cater to its theoretical concern: density, fertility and turbulence (Epstein and Axtell, 1996; Mezias and Lant, 1994). Collaborative governance research, by contrast, often does not require physical environment.

Whatever the nature of an environment may be to the agents in the real world, environment can be designed as an omniscient agent as shown in Figure 1. A properly designed environment is capable of accommodating basic level agents, of asking tasks and demands, and of giving feedback after evaluating agents' performance. At a casual glance, an agent may not grasp the whole feature of the environment; but only interact with it locally as much as it is known to the agent.

Internal mechanism

Other than the three basic building blocks of ABM, it is worth considering the role of the internal mechanism built in a specific model as an auxiliary building block. The auxiliary building block affects how the model works to generate the resulting structure. As a methodological merit, ABM can grow apparently complex phenomenon out of a few simple rules and agents. So, in between the ensuing structure and basic building blocks lies the internal mechanism of the model. The internal mechanism specifies the way through which multiple agents, rules and environments generate simulation results. If various rules are designed for agents to interact heavily with each other, the internal mechanism of the model becomes more intractable. Intractability is apparent when the designer may not explain verbally why a certain simulation result came out the way it did [3]. The internal mechanism has varying levels of tractability amenable to control by modelers. Out of this, one may say that a model is more transparent when it is more tractable.

There is a trade-off between the generative power and the tractability of a model. By adding more agent properties and rules, the Schelling model, for instance, could generate more real and diverse results; however, it becomes more difficult to sort out the effect of each property or rule. After all, it may be noted that although a researcher may not directly control the internal mechanism of the model, he or she can design the model in such a way to take control of the level of the tractability. Typically it is by using a stepwise or experimental design through which results are compared.

General social sciences

With the methodological potential of ABM espoused as such, we are ready to go to another important question raised: what to grow/simulate. Some early works took a population ecological perspective. Along this line, we can list up, in addition to the Schelling model, such works as “sugarscape” model (Epstein and Axtell, 1996), NK model (Rivkin and Siggelkow, 2003) and rugged fitness landscape model (Levinthal, 1997). The basic idea behind these models is that agents interact with each other and with the place they are located in, while pursuing their own goals. In the sugarscape model, agents are made akin to human beings alive in the food gathering economics era; in the NK model with the rugged fitness landscape, agents search for an optimal place for survival roaming across the landscape. The landscape changes continuously in response to agent action while simulating interconnectedness of the elements is active.

Another stream of ABM research heeded topics such as organizational learning, decision-making and structuration. For example, Cohen et al. (1972) developed the garbage can model. March (1991) depended on a work of simulation when devising a theoretical framework to examine exploration and exploitation as two modes of organizational learning. Carley's serial works (1991, 1992, 1995) on evolution of organizational structure and performance set a cornerstone in the field of computational organization theory.

Still more, ABM has proven useful in collaboration research. Axelrod (1997) simulated competition and collaboration to probe into evolution of new strategies, promotion of social norms and dissemination of culture. Bellamine-Ben Saoud and Mark (2007) applied an agent-based computer simulation to a new form of synchronous real-time collaborative engineering design. It is an attempt showing possibility of creating a visual artifact with cooperating agents interacting in a dynamic environment. Son and Rojas (2011) used ABM simulation to trace evolution of collaboration in temporary project teams. It shows the collaborative practices among heterogeneous individuals demonstrating varying work performance, depending on the number of individuals congregated to accomplish the project objective. Arvitrida, Robinson and Tako (2015) used ABM to model business competition and collaboration in supply chains. Their finding claims that the simple representation of a strategic landscape influences the competitive and collaborative behaviors in business.

Public administration research

Now, turning to public administration, we see only a handful of ABM employed research in the field. Seminal works are spotted in the field of simulation of collaborative governance and network. Johnston et al. (2011) published one of the earliest ABM works in the field. Their features centered on the effect of inclusion practices on stakeholders' expectation of each other's contribution and collaboration. Robertson and Choi (2012) and Choi and Robertson (2014a, b, 2019) published a series of ABM-supported theoretical articles. The mainline theme covered deliberation and decision in collaborative governance, focused on motivation, power and decision-making structure. More recently, Scott et al. (2019) performed a simulating study, dealing with stakeholder agreement in collaborative governance under varying conditions. In somewhat different vein, Maroulis and Wilensky (2015) applied an ABM method to investigate street-level implementation of innovation, drawing on organizational learning theory.

Outlook

Looking ahead for the areas where ABM could be fruitfully employed in public administration, we may wade through various disciplinary grounds. At this point, we have identified protruding mainline thematic topics centered on systemic inquiry into interactions between human behavior and structure. ABM opens a way to support this thematic pillar with the promise to improve relevant methodological frameworks and theoretical propositions.

First, as having been implied earlier, institutional analysis may gain momentum, when equipped with ABM (Siddiki et al., 2019). The benefit would be to enrich theoretical body of institutionalism, capable to display connection between agency and structure. This is a demanding task. When successful, it could render a powerful model to demonstrate pieces of institutional concepts and languages integrated into a consistent theoretical construct.

Second, ABM offers possibility to expand investigative width of hybrid governance and allow predicting its performance. Scholars using ABM would seek varying forms of hybrid governance designed by different combinations of ownership, rewards and authority crisscrossing market and hierarchy (Makadok and Coff, 2009, p. 299). When effectively explored, numerous prototype devices can be brought out and diversify relevant theoretical expositions.

Thirdly, recent findings of representative bureaucracy argue that symbolic representation takes varying effects. It becomes manifest when expressed in terms of response of actors to values of trust, fairness and performance of the organization (Grimmelikhuijsen et al., 2017, p. 52). To this end, ABM can be a serving tool to upscale learning models that have dealt with organizational diversity and performance.

Finally, network analysis regarding spread of information and knowledge, disease, and innovation has formed a large volume of research in the general computational field. ABM has the potential to regenerate, expand and put into scrutiny the relevant thematic arguments raised as they address the issue of the emergence and change of structure out of agent interactions. The potential would gain weight in public administration when bolstered by the accumulated or ongoing studies on spread of organizational reputation, corrupt behavior, policy idea or information and communication under crisis.

In summary, ABM has been in use for studies in wide-ranging topics. They range from population change, learning, adaptation, networking, decision-making to collaboration, particularly in simulating collaborative governance in public administration. The looming themes are inviting. It indicates disciplinary competence accorded on public administration research, made possible by access to the utilities of ABM. Not to mention holding a prominent outlook, ABM might be tapped by scholars in prevalent or prospective research on institutional analysis, hybrid governance, organizational diversity and network spread, to name a few.

Relationship to empirical findings

It is noteworthy that simulation results from an agent-based model are by no means a substitute for empirical findings. Researchers may be inclined to discussing simulation results as if they were as real as seen in empirical findings. It is especially the case when seeking practical implications from the executed model. Simulation results, even combined with empirical data, are but an extension of logic. Researchers, therefore, should be careful when trying to extract practical implications from a simulation result-especially when exploring new areas of research in the absence of empirical evidence to verify the implications to ensue.

Even with the limitation exhibited, however, it may be noted that simulation results and empirical findings are complementary to each other in two ways. First, an agent behavior can be modeled based on well-replicated empirical findings as well as game theoretic logic. This in particular applies when a researcher probes into social psychological issues such as human behaviors in an empirical public goods game (Choi and Robertson, 2019). Second, propositions or hypotheses derived from a model can motivate empirical researchers to validate empirical findings, and add rich nuance to them. This is a feature and a potential of ABM to reinforce the conclusion derived from a study, and lighting up avenues for the future course of research.

Topic selection

An enticing motivation for researchers to approach ABM is to simulate any human and social phenomena. Logically, there is no limit to modeling social entities as far as the modeler provides a legitimate rationale. This does not, however, downplay the importance that a decision on what to model and on how wide a range to cover by the model should be made in such a way to reconcile the rigor of the model. First, some topics may be simply addressed by empirical research than by ABM. For example, whether people are better motivated by money, gift or compliment (Gneezy and List, 2013) is definitely in the realm of empirical research. Second, there are ethical issues involving mostly simulating human behaviors or prototypes of human society based on biased real-world data, which are often found in artificial intelligence-driven decision making. When defining an agent, for instance, researchers may be warned so that they avoid stereotyping a certain group of people. Also, researchers may need to be aware of the social implications derived from simulation results. Ethical issues become explicit when the model is designed specifically for controlling human behaviors.

It also noteworthy that the rigor of a model can diminish when one attempts to simulate too wide a range of social structure. It may be difficult and costly, if not impossible, to simulate communication patterns of the entire Department of Homeland Security. Simply put, there is no point to simulate exactly the real world, just like there is no point to survey all American voters to predict a presidential election.

The limitation in topic selection, however, should not be confused with pioneering efforts in scholars' fields of research. For example, emotion is a fascinating issue in general social sciences, from psychology to public administration (Hattke et al., 2020; Levitats et al., 2019). ABM researchers have a full reason to design a model to address emotion. In addition, there are human behaviors not simply driven by “computational” reasoning. We need to be fully aware of what it would ontologically and epistemologically mean to simulate emotion. It would be particularly so when devising a “computational” information-processing model, not to mention how one can actually design an agent of emotion or “irrationality.”

Mixed method approach

An effective way to mitigate the limitations of ABM is to design a mixed-method approach from the beginning. Using a mixed method has been on the rise over the past (Mele and Belardinelli, 2019). One way to the mixed method is to devise a model that can reproduce the data compatible with the empirical data, when deployed on the relevant macro-level structure/dependent variable (Epstein, 1999), then run the model with an extended parameter space to further develop a theory. Another one is to first devise a model and get a set of results from it, and then combine it with typical empirical cases that represent or enrich each type of results. One can even combine ABM with meta-analysis to scan and enlarge the picture of the research arena. Maroulis and Wilensky's work (2015) on social and task interdependencies in innovation is an exemplary work combining empirical data with ABM.

Extended use of sensitivity analysis

Sensitivity analysis is a standard, or often a routine required of articles employing ABM. It can be an effective tool when carefully combined with the mixed method. According to Epstein (1999, p. 52), sensitivity is about the effect of small changes in input such as initial parameter values on output such as performance of the simulated system. Although sensitivity analysis is considered to demonstrate the rigor of simulation results out of rather arbitrarily defined parameter values, more strategic uses of sensitivity analysis may add values to ABM.

For one thing, sensitivity analysis can enhance the chance for ensuring replicability. Although the recent academic practice asks authors to provide supplemental materials such as (pseudo-) code of their model in the article or a public website [4], the point is not so much to let other researchers run the same model with the same language. It is to replicate the similar-though the same theoretical insights held-results with different models coded by different languages and algorithms. Harrison et al. (2007, p. 1243) well put the point:

[F]ailure to replicate a finding does not necessarily mean that it is wrong, however. It may be that the original finding holds only under certain conditions or only for certain ways of operationalizing the formal model.

Sensitivity analysis, in this sense, offers a sort of “confidence interval” from which to tell whether two or more different models are compatible with each other.

Multidisciplinary research projects

Contributions from ABM may be maximized when a researcher designs and performs a multidisciplinary research project that are not easily amendable to empirical research. Note that an agent-based model is built from a set of micro-level assumptions about agents and their interactions. It, then, simulates a macro-level structure theoretically expected to be linked with the former. As such, it is not surprising that social psychologists find ABM useful in interdisciplinary studies. Smith and Conrey (2007, p. 101) put it:

One of the primary features of ABM is that it allows, even forces, theoretical thinking to cross levels, as modelers seek to understand high-level structures and processes as outcomes of low-level agent interactions. Thus, ABM provides a common framework for processes at multiple levels, making it a natural focus for cross-disciplinary integration.

Scholars often criticize that laboratory experiments are carried out in an isolated condition, making it impossible to investigate contextual influence (Borgatta and Bohrnstedt, 1974). Although ABM cannot substitute empirical experiments, it can still empower the contextual implications, offering a multi-level analytic tool. In this sense, ABM is an investigative muscle to buttress multidisciplinary research, which can move forwards to help behavioral foundation taking shape in public administration research.

AI and machine learning

Public administration research is not an exception in benefiting from the rich pool of public data and machine learning. The trend is in parallel with the rising interest in machine learning we see these days (Anastasopoulos and Whitford, 2019). Given that an agent is a stylized computational entity of human behavior, it would not be amiss to expect artificial intelligence technique and machine learning to meet with ABM. The congruence is expected to come in a near future and open up a new horizon of intellectual inquiry. The immediate venue would probably be in the task of researchers to design and “educate” agents to act more like a human being. It would be a dramatic shift exhibiting artificial intelligence algorithms, overriding the “manually coded” programs that simply follow a set of tailored rules.