Cloud security is of paramount importance in the new era of computing. Advanced malware can hide their behavior on detection of the presence of a security tool at a tenant virtual machine (TVM). Hence, TVM-layer security solutions are not reliable. In this paper, we propose a Virtual Machine Introspection (VMI) based security architecture design for fine granular monitoring of the virtual machines to detect known attacks and their variants. We have developed techniques for monitoring the TVMs at the process level and system call level to detect attacks such as those based on malicious hidden processes, attacks that disable security tools in the virtual machines and attacks that alter the behavior of legitimate applications to access sensitive data. Our architecture, VMGuard, utilizes the introspection feature at the VMM-layer to analyze system call traces of programs running on TVM. VMGuard applies the software breakpoint injection technique which is OS agnostic and can be used to trap the execution of programs. Motivated by text mining approaches, VMGuard provides ‘Bag of n-grams (BonG)’ approach integrated with Term Frequency-Inverse Document Frequency (TF-IDF) method, to extract and select features of normal and attack traces. It then applies the Random Forest classifier to produce a generic behavior for different categories of intrusions of the monitored TVM. We have implemented a prototype and conducted a detailed analysis using University of New Mexico (UNM) datasets and a Windows malware dataset obtained from the University of California. The results obtained are promising and demonstrate the applicability of the VMGuard. We compare VMGuard with existing techniques and discuss its advantages.

中文翻译：

---

VMGuard：一种基于 VMI 的云环境入侵检测安全架构

在新的计算时代，云安全至关重要。高级恶意软件可以在检测到租户虚拟机 (TVM) 上是否存在安全工具时隐藏其行为。因此，TVM 层安全解决方案并不可靠。在本文中，我们提出了一种基于虚拟机自省 (VMI) 的安全架构设计，用于对虚拟机进行细粒度监控，以检测已知攻击及其变体。我们开发了在进程级别和系统调用级别监控 TVM 的技术，以检测基于恶意隐藏进程的攻击、禁用虚拟机中安全工具的攻击以及改变合法应用程序行为以访问敏感数据的攻击. 我们的架构，VMGuard，利用 VMM 层的自省功能来分析在 TVM 上运行的程序的系统调用跟踪。VMGuard 应用软件断点注入技术，该技术与操作系统无关，可用于捕获程序的执行。受文本挖掘方法的启发，VMGuard 提供了与术语频率-逆文档频率 (TF-IDF) 方法集成的“n-gram 袋 (BonG)”方法，以提取和选择正常和攻击痕迹的特征。然后应用随机森林分类器为受监控 TVM 的不同类别的入侵生成通用行为。我们已经实现了一个原型，并使用新墨西哥大学 (UNM) 数据集和从加利福尼亚大学获得的 Windows 恶意软件数据集进行了详细分析。获得的结果很有希望并证明了 VMGuard 的适用性。我们将 VMGuard 与现有技术进行比较并讨论其优势。