Software development required for constructing multi-agent systems (MAS) usually becomes challenging and time-consuming due to the properties of autonomy, distributedness, and openness of these systems in addition to the complicated nature of internal agent behaviors and agent interactions. To facilitate MAS development, the researchers propose various domain-specific modeling languages (DSMLs) by enriching MAS metamodels with a defined syntax and semantics. Although the descriptions of these languages are given in the related studies with the examples of their use, unfortunately, many are not evaluated in terms of either the usability (being hard to learn, understand and use) or the quality of the generated artifacts. Hence, in this paper, we introduce an evaluation framework, called AgentDSM-Eval, with its supporting tool which can be used to evaluate MAS DSMLs systematically according to various quantitative and qualitative aspects of agent software development. The empirical evaluation, presented by the AgentDSM-Eval framework, was successfully applied for one of the well-known MAS DSMLs. The assessment showed that both MAS domain coverage of DSMLs and the agent developers’ adoption of modeling elements can be determined with this framework. Moreover, the tool's quantitative results can assess MAS DSML's performance on the development time and throughput. AgentDSM-Eval also enables the qualitative assessment of MAS DSML features according to novel quality characteristics and measures, which it defines specifically for the MAS domain.

除了内部代理行为和代理交互的复杂性之外，由于这些系统的自治性，分布性和开放性，构建多代理系统（MAS）所需的软件开发通常变得具有挑战性和耗时。为了促进MAS的发展，研究人员通过使用定义的语法和语义丰富MAS元模型，提出了各种领域特定的建模语言（DSML）。尽管在相关研究中给出了这些语言的描述及其使用示例，但不幸的是，许多语言都没有在可用性（难以学习，理解和使用）或生成的工件的质量方面进行评估。因此，在本文中，我们介绍了一个称为AgentDSM-Eval的评估框架，其支持工具可用于根据代理软件开发的各种定量和定性方面来系统地评估MAS DSML。由AgentDSM-Eval框架提供的经验评估已成功应用于一种著名的MAS DSML。评估表明，可以使用此框架确定DSML的MAS域覆盖范围和代理开发人员对建模元素的采用。此外，该工具的定量结果可以评估MAS DSML在开发时间和吞吐量方面的性能。AgentDSM-Eval还可以根据新颖的质量特征和度量对MAS DSML功能进行定性评估，这是专门针对MAS域定义的。