Context : Recent papers have proposed the use of grey literature (GL) and multivocal reviews. These papers have raised issues about the practices used for systematic reviews (SRs) in software engineering (SE) and suggested that there should be changes to the current SR guidelines. Objective : To investigate whether current SR guidelines need to be changed to support GL and multivocal reviews. Method : We discuss the definitions of GL and the importance of GL and of industry-based field studies in SE SRs. We identify properties of SRs that constrain the material used in SRs: a) the nature of primary studies; b) the requirements of SRs to be auditable, traceable, and reproducible; and explain why these requirements restrict the use of blogs in SRs. Results : SR guidelines have always considered GL as a possible source of primary studies and have never supported exclusion of field studies that incorporate the practitioners’ viewpoint. However, the concept of GL, which was meant to refer to documents that were not formally published, is now being extended to information from sources such as blogs/tweets/Q&A posts. Thus, it might seem that SRs do not make full use of GL because they do not include such information. However, the unit of analysis for an SR is the primary study. Thus, it is not the source but the type of information that is important. Any report describing a rigorous empirical evaluation is a candidate primary study. Whether it is actually included in an SR depends on the SR eligibility criteria. However, any study that cannot be guaranteed to be publicly available in the long term should not be used as a primary study in an SR. This does not prevent such information from being aggregated in surveys of social media and used in the context of evidence-based software engineering (EBSE). Conclusions : Current guidelines for SRs do not require extensions, but their scope needs to be better defined. SE researchers require guidelines for analysing social media posts (e.g., blogs, tweets, vlogs), but these should be based on qualitative primary (not secondary) study guidelines. SE researchers can use mixed-methods SRs and/or the fourth step of EBSE to incorporate findings from social media surveys with those from SRs and to develop industry-relevant recommendations.

中文翻译：

---

软件工程副业应如何纳入灰色材料？

背景：最近的论文提出使用灰色文献 (GL) 和多声部评论。这些论文提出了有关软件工程 (SE) 中用于系统审查 (SR) 的实践的问题，并建议应对当前的 SR 指南进行更改。目标：调查是否需要更改当前的 SR 指南以支持 GL 和多声部评论。方法：我们讨论了 GL 的定义以及 GL 和基于行业的实地研究在 SE SR 中的重要性。我们确定限制 SR 中使用的材料的 SR 属性：a) 初步研究的性质；b) SR 的要求是可审核的、可追溯的和可复制的；并解释为什么这些要求限制在 SR 中使用博客。结果：SR 指南一直将 GL 视为原始研究的可能来源，并且从不支持排除包含从业者观点的实地研究。然而，GL 的概念原本是指未正式发布的文档，现在正在扩展到来自博客/推文/问答帖子等来源的信息。因此，SR 似乎没有充分利用 GL，因为它们不包含此类信息。但是，SR 的分析单元是主要研究。因此，它不是来源，但重要的信息类型。任何描述严格实证评估的报告都是候选初步研究。它是否实际包含在 SR 中取决于 SR 资格标准。但是，任何不能保证长期公开的研究都不应用作 SR 中的主要研究。这并不妨碍此类信息在社交媒体调查中汇总，并在循证软件工程 (EBSE) 的背景下使用。结论：当前的 SR 指南不需要扩展，但需要更好地定义它们的范围。SE 研究人员需要分析社交媒体帖子（例如博客、推文、视频博客）的指南，但这些应该基于定性的主要（而非次要）研究指南。SE 研究人员可以使用混合方法 SR 和/或 EBSE 的第四步，将社交媒体调查的结果与 SR 的结果结合起来，并制定与行业相关的建议。