On the use of the analytic hierarchy process in the evaluation of domain-specific modeling languages for multi-agent systems
Introduction
Software agents and Multi-agent Systems (MASs) composed by these agents are used effectively in the design and implementation of various autonomous and complex intelligent systems [1], [2], [3], [4]. However, autonomous, reactive and pro-active behaviors of agents make it more difficult to develop agent-based software systems than other systems. As also indicated in [5], software systems, made up of a possibly millions of agents need to be developed in near future where agents are able to behave in a reliable, intelligent, and trustworthy way at any scale of observation. Modeling both the open nature of MAS and its relation with the outer environment is still challenging. The implementation of the interactions between agents showing autonomous and proactive behavior require a more extensive and complicated program development including verification, data integrity, resource management and concurrency of these agents [6]. While programming agent plans or rules, the mechanisms for selecting and achieving goals as well as selecting and scheduling the agent intentions that are applicable in all situations of agent execution should be constructed properly. Besides, the communication of agents within MAS organizations and the composition of agent behaviors become even more complicated and challenging to implement while the varying needs of different agent environments are considered [7]. Hence, it is important to work at higher levels of abstraction prior to coding agent software during the development of MASs [8].
In order to solve the aforementioned problems of MAS development, the researchers working in the field of Agent-oriented Software Engineering (AOSE) define a variety of meta-models that incorporate the basic components and relationships of agent models [9]. By enriching these meta-models with specific syntactic and semantic definitions, the researchers also propose various domain-specific languages (DSLs) [10], [11] or domain-specific modeling languages (DSMLs) [12] to facilitate model-driven engineering (MDE) of agents (e.g. [8], [13], [14], [15], [16], [17], [18]). These modeling languages, which are shortly referred as MAS DSMLs hereinafter, may leverage the abstraction, expression power and ease of use in software development, since MAS models can be defined visually with these DSMLs first and then the software required for the related MAS can be automatically generated via a series of transformations defined on these models. In addition, most of these languages provide both static and dynamic modeling of agent software through various perspectives such as internal agent behavior, interactions with other agents and the use of other environment sources. Hence, the modeling domain of these DSMLs cover the whole MAS including all of the agent components indicated above. Due to the capacity of identifying the essence of a MAS using visual abstractions which are close to the agent domain elements, it is expected that new DSMLs will also be introduced in the near future to address a variety of concerns in MAS [19].
Although the descriptions of the MAS DSMLs are given in the studies with including some examples of how they can be utilized during MAS development, unfortunately many of these studies do not consider a through evaluation of the proposed language, e.g. evaluating the usability of the language and the efficiency of the generated artifacts. Remaining studies (e.g. [8], [16], [20], [21]) cover some sort of quantitative and/or qualitative language assessment in an idiosyncratic manner, i.e. they only aim at the evaluation of some specific features of one single MAS DSML and hence they lack providing a general evaluation method for MAS DSMLs. More importantly they do not support the comparative evaluation of many MAS DSMLs altogether. Such a comparative evaluation may help agent software developers to choose the MAS DSML which fits well into the requirements of their MAS software development processes.
Multiple-criteria Decision Making (MCDM) techniques help decision-makers to choose the best among many alternatives in environments with different competitors and often conflicting criteria. Over the past few years, the consistent growth has been observed in the literature on MCDM, while the Analytical Hierarchical Process (AHP) [22] keeps the position which is the most frequently used MCDM technique. The AHP has been proposed to build multiple-criteria decision support method via the prioritization of each alternative based on the pairwise comparisons of each object with all the others. The popularity of AHP in the decision-making approach derives from its ability to form MCDM that includes qualitative data. The method uses a decision matrix obtained by making pairwise comparisons of the linguistic or discrete presentation of the information. Generally, it seeks expert opinions to get the information about decision alternatives. The AHP, which is often used in solving complicated decision-making problems in other areas (e.g. [23], [24], [25]) as well as in software engineering (e.g. [26], [27], [28], [29], [30]) may also contribute to the assessment of MAS DSMLs. Hence, in this paper, we introduce an AHP-based methodology which provides the comparative evaluation of MAS DSMLs according to many language criteria prioritized by the agent software developers and leads the selection of the best MAS DSML among the alternatives. Main contributions of the study can be listed as follows:
* A well-defined and categorized set of criteria and sub-criteria is presented first time which can be used to create AHP hierarchy models for the comparative evaluation of various MAS DSMLs.
* With conforming to the AHP steps, a tool assisted evaluation methodology is constructed which consists of determining MAS DSML alternatives, prioritization of both evaluation criteria and MAS DSML alternatives according to these criteria, automatic calculation of the importance distributions and finally the suggestion of the best DSML among the alternatives.
* Based on the voluntarily participation of a group of agent software developers, the proposed methodology was applied for the comparative evaluation of four MAS DSMLs widely used in AOSE.
The rest of the paper is organized as follows: Section 2 gives the related work. Section 3 briefly describes the AHP for the background. Section 4 presents the AHP-based MAS DSML evaluation methodology. Section 5 discusses the experiment for comparing MAS DSML alternatives as well as the results achieved from applying the methodology. Section 6 concludes the paper.
Section snippets
Related work
In the early 2000s, use of UML-based languages, such as AUML [31] and AML [32] became popular in MAS modeling. However, relying too much on UML which is originally designed for object-oriented system specification as well as lacking both formal and/or operational semantics and up-to-date tools led to the decline of agent developers’ interest in these languages over time [9], [33]. Following these first efforts, the researchers have made and are still making significant contribution on the
The analytic hierarchy process
One of the most frequently used approaches to MCDM problems in which factors are organized in a hierarchical structure is the Analytic Hierarchy Process (AHP) [22], [54]. The AHP derives priority scales through pairwise comparisons of preferences [55]. In this section, we briefly introduce the AHP on which our evaluation methodology is based. For readers who are not familiar with AHP, Table 1 describes each fundamental AHP matrix and vector. To solve a decision problem by AHP method, following
The AHP-based evaluation methodology for MAS DSMLs
This section introduces the methodology which we propose for the comparative evaluation of MAS DSMLs. As conforming to the AHP steps described in the previous section, the application of this evaluation methodology consists of defining MAS DSML evaluation criteria and determining MAS DSML alternatives, collecting and preparing data for the evaluation, calculating weights, and finally analyzing the achieved results. Fig. 1 illustrates the overview of this methodology. Definition of the
Experiment and discussion
The methodology was used for the comparative evaluation of four different MAS DSMLs. In the following subsections, determining these MAS DSML alternatives, collecting and preparing data, calculating the weights and finally analyzing the results are discussed.
Conclusion
An AHP-based evaluation methodology for the comparative evaluation of MAS DSMLs has been introduced in this paper. For this purpose, a categorized set of criteria which can be used for the multi-criteria decision making has been defined. These criteria can be prioritized by the agent developers according to their modeling language expectations and the application of the methodology allows the evaluation of MAS DSML alternatives based on this prioritization. As the result of the automatic
CRediT authorship contribution statement
Tansu Zafer Asici: Investigation, Software, Methodology, Visualization, Writing - original draft. Baris Tekin Tezel: Conceptualization, Methodology, Validation, Writing - original draft. Geylani Kardas: Conceptualization, Methodology, Supervision, Writing - original draft, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
This study was funded by the Scientific and Technological Research Council of Turkey (TÜBİTAK) under grant 115E591.
References (68)
Negotiation and cooperation between agents for generalizing geographic objects
J. Comput. Lang.
(2019)- et al.
Domain-specific languages: A systematic mapping study
Inf. Softw. Technol.
(2016) - et al.
On the use of a domain-specific modeling language in the development of multiagent systems
Eng. Appl. Artif. Intell.
(2014) - et al.
MAS-ML 2.0: Supporting the modelling of multi-agent systems with different agent architectures
J. Syst. Softw.
(2015) - et al.
Sam: a tool to ease the development of intelligent agents
Eng. Appl. Artif. Intell.
(2017) - et al.
Special issue on model-driven engineering of multi-agent systems in theory and practice
Comput. Lang. Syst. Struct.
(2017) - et al.
Agent-oriented model-driven development for JADE with the JADEL programming language
Comput. Lang. Syst. Struct.
(2017) - et al.
Analytic hierarchy process: An overview of applications
European J. Oper. Res.
(2006) - et al.
Evaluation of clustering algorithms for financial risk analysis using MCDM methods
Inform. Sci.
(2014) - et al.
Designing an integrated AHP based decision support system for supplier selection in automotive industry
Expert Syst. Appl.
(2016)
A novel approach to evaluate software vulnerability prioritization
J. Syst. Softw.
User feature-aware trustworthiness measurement of cloud services via evidence synthesis for potential users
J. Vis. Lang. Comput.
Investigation on test effort estimation of mobile applications: Systematic literature review and survey
Inf. Softw. Technol.
Model-driven engineering techniques for the development of multi-agent systems
Eng. Appl. Artif. Intell.
A technique for defining agent-oriented engineering processes with tool support
Eng. Appl. Artif. Intell.
Business and model-driven development of BDI multi-agent systems
Neurocomputing
User-story driven development of multi-agent systems: A process fragment for agile methods
Comput. Lang. Syst. Struct.
An analytic hierarchy process and two-sided matching based decision support system for military personnel assignment
Inform. Sci.
Project selection for oil-fields development by using the AHP and fuzzy TOPSIS methods
Expert Syst. Appl.
Applying a selection method to choose quality attribute techniques
Inf. Softw. Technol.
Toward automated feature model configuration with optimizing non-functional requirements
Inf. Softw. Technol.
A novel TOPSIS evaluation scheme for cloud service trustworthiness combining objective and subjective aspects
J. Syst. Softw.
How to make a decision: the analytic hierarchy process
European J. Oper. Res.
Usability analysis of visual programming environments: A ’cognitive dimensions’ framework
J. Vis. Lang. Comput.
Cognitive dimensions: Achievements, new directions, and open questions
J. Vis. Lang. Comput.
Recent advances in consensus of multi-agent systems: A brief survey
IEEE Trans. Ind. Electron.
Multi-agent system for microgrids: design, optimization and performance
Artif. Intell. Rev.
Challenges and research directions in agent-oriented software engineering
Auton. Agents Multi-Agent Syst.
Designing a source-level debugger for cognitive agent programs
Auton. Agents Multi-Agent Syst.
Engineering multi-agent systems: State of affairs and the road ahead
ACM SIGSOFT Softw. Eng. Notes
Domain-specific modelling language for belief–desire–intention software agents
IET Softw.
Model-driven development of multi-agent systems: a survey and evaluation
Knowl. Eng. Rev.
When and how to develop domain-specific languages
ACM Comput. Surv.
Cited by (8)
An executable domain-specific modeling language for simulating organizational auction-based coordination strategies for crisis response
2024, Simulation Modelling Practice and TheoryValidation and verification in domain-specific modeling method engineering: an integrated life-cycle view
2023, Software and Systems Modeling