Conducting technology-oriented experiments (i.e., experiments in which treatments are applied to objects by a computer-based tool) without proper tool support is often a time-consuming and highly error-prone task. Although many techniques have been proposed to help conducting controlled experiments, none of them simultaneously addresses (1) the executable specification of experiments at a high level of abstraction; (2) automated treatment execution and automated data analysis from the experiment specification; and (3) formal guaranties of the correctness of results according to an experiment specification for technology-oriented experiments. To address these issues, we provide a Domain-Specific Modeling approach to create a Web-based tool (ExpRunA ) comprising a Domain-Specific Language named ToExpDSL , execution and analysis script generators, a supporting framework, and a running infrastructure. An experimenter uses ToExpDSL to specify an experiment using experimentation concepts. From this specification, applications corresponding to the underlying treatments are executed, execution results are collected and analyzed, and, finally, the analysis results are presented to the experimenter. We establish the consistency of such results with respect to the experiment specification by formalizing and proving key correctness properties of ExpRunA . We empirically evaluated ExpRunA with respect to automation by replicating three already published experiments; we evaluated the level of abstraction by a qualitative assessment. Our empirical evaluation shows that ToExpDSL is expressive enough to specify three technology-oriented experiments and that ExpRunA can be used to enable sound automation of execution and analysis from the specification of technology-oriented experiments at a high level of abstraction.

中文翻译：

---

ExpRunA：面向技术的实验的特定领域方法

在没有适当工具支持的情况下进行面向技术的实验（即，通过基于计算机的工具对对象进行处理的实验）通常是一项耗时且容易出错的任务。尽管已经提出了许多技术来帮助进行受控实验，但它们都不能同时解决（1）高度抽象的实验可执行规范；（2）根据实验规范自动执行治疗并进行自动数据分析；（3）根据面向技术实验的实验规范对结果正确性进行形式上的保证。为了解决这些问题，我们提供了一种特定于领域的建模方法来创建一个基于Web的工具（ExpRunA），该工具包含名为 ToExpDSL，执行和分析脚本生成器，支持框架以及正在运行的基础结构。实验人员使用 ToExpDSL通过实验概念指定实验。根据该规范，执行与基础处理相对应的应用程序，收集并分析执行结果，最后将分析结果呈现给实验人员。通过形式化和证明ExpRunA的关键正确性，我们建立了与实验规范相关的结果一致性 。我们根据经验评估了ExpRunA通过复制三个已经发布的实验进行自动化；我们通过定性评估来评估抽象水平。我们的经验评估表明，  ToExpDSL具有足够的表现力，可以指定三个面向技术的实验，并且可以使用ExpRunA从高度抽象的面向技术的实验规范中实现执行和分析的合理自动化。