当前位置: X-MOL 学术Inf. Softw. Technol. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Processes, challenges and recommendations of Gray Literature Review: An experience report
Information and Software Technology ( IF 3.9 ) Pub Date : 2021-05-03 , DOI: 10.1016/j.infsof.2021.106607
He Zhang , Runfeng Mao , Huang Huang , Qiming Dai , Xin Zhou , Haifeng Shen , Guoping Rong

Context:

Systematic Literature Review (SLR), as a tool of Evidence-Based Software Engineering (EBSE), has been widely used in Software Engineering (SE). However, for certain topics in SE, especially those that are trendy or industry driven, academic literature is generally scarce and consequently Gray Literature (GL) becomes a major source of evidence. In recent years, the adoption of Gray Literature Review (GLR) or Multivocal Literature Review (MLR) is rising steadily to provide the state-of-the-practice of a specific topic where SLR is not a viable option.

Objective:

Although some SLR guidelines recommend the use of GL and several MLR guidelines have already been proposed in SE, researchers still have conflicting views on the value of GL and commonly accepted GLR or MLR studies are generally lacking in terms of publication. This experience report aims to shed some light on GLR through a case study that uses SLR and MLR guidelines to conduct a GLR on an emerging topic in SE to specifically answer the questions related to the reasons of using GL, the processes of conducting GL, and the impacts of GL on review results.

Method:

We retrospect the review process of conducting a GLR on the topic of DevSecOps with reference to Kitchenham’s SLR and Garousi’s MLR guidelines. We specifically reflect on the processes we had to adapt in order to tackle the challenges we faced. We also compare and contrast our GLR with existing MLRs or GLRs in SE to contextualize our reflections.

Results:

We distill ten challenges in nine activities of a GLR process. We provide reasons for these challenges and further suggest ways to tackle them during a GLR process. We also discuss the decision process of selecting a suitable review methodology among SLR, MLR and GLR and elaborate the impacts of GL on our review results.

Conclusion:

Although our experience on GLR is mainly derived from a specific case study on DevSecOps, we conjecture that it is relevant and would be beneficial to other GLR or MLR studies. We also expect our experience would contribute to future GLR or MLR guidelines, in a way similar to how SLR guidelines learned from the SLR experience report a dozen years ago. In addition, other researchers may find our decision making process useful before they conduct their own reviews.



中文翻译:

《灰色文学评论》的过程,挑战和建议:经验报告

语境:

作为基于证据的软件工程(EBSE)的工具,系统文献综述(SLR)已在软件工程(SE)中得到广泛使用。但是,对于SE中的某些主题,尤其是那些由时尚或行业驱动的主题,学术文献通常是稀缺的,因此,灰色文学(GL)成为了主要的证据来源。近年来,灰色文学评论(GLR)或多声文学评论(MLR)的采用正在稳步增长,以提供特定主题的实践状态,而单反不是可行的选择。

客观的:

尽管一些SLR指南建议使用GL,并且SE中已经提出了一些MLR指南,但研究人员对GL的价值仍存在分歧,并且在发表方面通常缺乏公认的GLR或MLR研究。这份经验报告旨在通过案例研究阐明GLR,该案例研究使用SLR和MLR指南对SE中的一个新兴主题进行GLR,以专门回答与使用GL的原因,进行GL的过程有关的问题,以及GL对审核结果的影响。

方法:

我们回顾了针对KitchenSham的SLR和Garousi的MLR指南进行的关于DevSecOps主题的GLR的审查过程。我们特别思考必须采取的流程,以应对所面临的挑战。我们还将SE中的GLR与现有的MLR或GLR进行比较和对比,以体现我们的观点。

结果:

我们在GLR流程的九项活动中提炼出十项挑战。我们提供了这些挑战的原因,并进一步提出了在GLR流程中解决这些挑战的方法。我们还将讨论在SLR,MLR和GLR中选择合适的审查方法的决策过程,并详细说明GL对我们的审查结果的影响。

结论:

尽管我们在GLR方面的经验主要来自于DevSecOps的特定案例研究,但我们推测它是相关的,并且对其他GLR或MLR研究是有益的。我们还希望我们的经验可以为未来的GLR或MLR指南做出贡献,其方式类似于十几年前从SLR经验报告中学到的SLR指南。此外,其他研究人员在进行自己的评论之前可能会发现我们的决策过程很有用。

更新日期:2021-05-05
down
wechat
bug