Context

Systematic Literature Reviews (SLRs) are frequently used to synthesize evidence in Software Engineering (SE), however replicating and keeping SLRs up-to-date is a major challenge. The activity of studies selection in SLR is labor intensive due to the large number of studies that must be analyzed. Different approaches have been investigated to support SLR processes, such as: Visual Text Mining or Text Classification. But acquiring the initial dataset is time-consuming and labor intensive.

Objective

In this work, we proposed and evaluated the use of Text Classification to support the studies selection activity of new evidences to update SLRs in SE.

Method

We applied Text Classification techniques to investigate how effective and how much effort could be spared during the studies selection phase of an SLR update. Considering the SLRs update scenario, the studies analyzed in the primary SLR could be used as a classified dataset to train Supervised Machine Learning algorithms. We conducted an experiment with 8 Software Engineering SLRs. In the experiments, we investigated the use of multiple preprocessing and feature extraction tasks such as tokenization, stop words removal, word lemmatization, TF-IDF (Term-Frequency/Inverse-Document-Frequency) with Decision Tree and Support Vector Machines as classification algorithms. Furthermore, we configured the classifier activation threshold for maximizing Recall, hence reducing the number of Missed selected studies.

Results

The techniques accuracies were measured and the results achieved on average a F-Score of 0.92 and 62% of exclusion rate when varying the activation threshold of the classifiers, with a 4% average number of Missed selected studies. Both the Exclusion rate and number of Missed selected studies were significantly different when compared to classifier which did not use the configuration of the activation threshold.

Conclusion

The results showed the potential of the techniques in reducing the effort required of SLRs updates.

中文翻译：

---

减少使用文本分类进行软件工程系统文献综述更新的工作

语境

系统文献评论（SLR）在软件工程（SE）中经常用于综合证据，但是复制和保持SLR为最新是一个重大挑战。由于必须分析大量的研究，因此在SLR中选择研究的工作量很大。为了支持SLR过程，已经研究了不同的方法，例如：可视文本挖掘或文本分类。但是获取初始数据集既费时又费力。

目的

在这项工作中，我们提出并评估了文本分类的使用，以支持新证据的研究选择活动，以更新SE中的SLR。

方法

我们应用了“文本分类”技术来调查在SLR更新的研究选择阶段如何有效和省下多少精力。考虑到SLR的更新场景，在主要SLR中分析的研究可以用作分类数据集，以训练监督机器学习算法。我们对8个软件工程SLR进行了实验。在实验中，我们使用决策树和支持向量机作为分类算法，研究了多种预处理和特征提取任务的使用，例如令牌化，停用词去除，词词形化，TF-IDF（术语频率/文档逆向） 。此外，我们配置了分类器激活阈值以最大程度地提高查全率，从而减少了未选中研究的数量。

结果

测量技术的准确性，当改变分类器的激活阈值时，平均F分数为0.92，排除率为62％，漏选研究的平均数为4％。与不使用激活阈值配置的分类器相比，排除率和错过的选定研究数均存在显着差异。

结论

结果显示了该技术在减少SLR更新所需的工作量方面的潜力。