Context: Neural Network (NN) algorithms have been successfully adopted in a number of Safety-Critical Cyber-Physical Systems (SCCPSs). Testing and Verification (T&V) of NN-based control software in safety-critical domains are gaining interest and attention from both software engineering and safety engineering researchers and practitioners.

Objective: With the increase in studies on the T&V of NN-based control software in safety-critical domains, it is important to systematically review the state-of-the-art T&V methodologies, to classify approaches and tools that are invented, and to identify challenges and gaps for future studies.

Method: By searching the six most relevant digital libraries, we retrieved 950 papers on the T&V of NN-based Safety-Critical Control Software (SCCS). Then we filtered the papers based on the predefined inclusion and exclusion criteria and applied snowballing to identify new relevant papers.

Results: To reach our result, we selected 83 primary papers published between 2011 and 2018, applied the thematic analysis approach for analyzing the data extracted from the selected papers, presented the classification of approaches, and identified challenges.

Conclusion: The approaches were categorized into five high-order themes, namely, assuring robustness of NNs, improving the failure resilience of NNs, measuring and ensuring test completeness, assuring safety properties of NN-based control software, and improving the interpretability of NNs. From the industry perspective, improving the interpretability of NNs is a crucial need in safety-critical applications. We also investigated nine safety integrity properties within four major safety lifecycle phases to investigate the achievement level of T&V goals in IEC 61508-3. Results show that correctness, completeness, freedom from intrinsic faults, and fault tolerance have drawn most attention from the research community. However, little effort has been invested in achieving repeatability, and no reviewed study focused on precisely defined testing configuration or defense against common cause failure.

上下文：神经网络（NN）算法已在许多安全关键的网络物理系统（SCCPS）中成功采用。在安全关键领域中基于NN的控制软件的测试和验证（T＆V）引起了软件工程和安全工程研究人员和从业人员的关注和关注。

目的：随着对安全关键领域基于NN的控制软件的T＆V的研究的增加，重要的是系统地审查最新的T＆V方法论，对发明的方法和工具进行分类，并确定未来研究的挑战和差距。

方法：通过搜索六个最相关的数字图书馆，我们检索了950篇基于NN的安全关键控制软件（SCCS）的T＆V论文。然后，我们根据预定义的包含和排除标准对论文进行了过滤，并通过滚雪球技术来识别新的相关论文。

结果：为达到结果，我们选择了2011年至2018年发表的83篇主要论文，应用主题分析方法来分析从选定论文中提取的数据，介绍了方法的分类并确定了挑战。

结论：这些方法分为五个高级主题，即确保神经网络的鲁棒性，提高神经网络的故障复原力，测量和确保测试的完整性，确保基于神经网络的控制软件的安全性以及提高神经网络的可解释性。从行业角度来看，在安全关键型应用程序中，提高神经网络的可解释性是至关重要的需求。我们还调查了四个主要安全生命周期阶段中的九个安全完整性属性，以调查IEC 61508-3中T＆V目标的实现水平。结果表明，正确性，完整性，无内在缺陷和容错性已引起了研究界的最大关注。但是，在实现可重复性方面的投入很少，