As the Internet rapidly develops, the types and quantity of malware continue to diversify and increase, and the technology of evading security software is becoming more and more advanced. This paper proposes a malware detection method based on deep learning, which combines malware visualization technology with convolutional neural network. The structure of neural network is based on VGG16 network. This paper proposes the hybrid visualization of malware, combining static and dynamic analysis. In hybrid visualization, we use the Cuckoo Sandbox to carry out dynamic analysis on the samples, convert the dynamic analysis results into a visualization image according to a designed algorithm, and train the neural network on static and hybrid visualization images. Finally, we test the performance of the malware detection method we propose, evaluating its effectiveness on detecting unknown malware.

随着Internet的快速发展，恶意软件的种类和数量继续多样化和增加，逃避安全软件的技术也越来越先进。本文提出了一种基于深度学习的恶意软件检测方法，该方法将恶意软件可视化技术与卷积神经网络相结合。神经网络的结构基于VGG16网络。本文提出了结合静态和动态分析的恶意软件混合可视化。在混合可视化中，我们使用Cuckoo Sandbox对样本进行动态分析，根据设计的算法将动态分析结果转换为可视化图像，并在静态和混合可视化图像上训练神经网络。最后，我们测试了我们提出的恶意软件检测方法的性能，