To explore the design of the virtual reality (VR) augmented reality (AR) mobile platform and game decision model based on deep learning (DL), the gesture-based interaction of VR games based on Leap Motion is researched. Based on the interactive features of gestures, a set of general gesture interaction rules in VR games is established. In the meantime, according to the theoretical basis and the characteristics of VR, a set of general models of VR gesture interaction is designed, the factors affecting the efficiency of VR gesture interaction are studied, and reasonable interaction feedback is designed. By using the computer vision and image processing technology, gesture-based interaction can collect natural gestures, extract gesture features, recognize gesture indications, and respond to the user demands. Also, it can extract the basic gestures from gesture-based interaction in VR, analyze the basic features of gestures and gesture-based interaction in VR games, and describes the gesture features by mathematical vectors and sets. The research results show that the application of gesture feature design method in the game can analyze the factors affecting the interaction efficiency. Also, the usability of the gesture-based interaction designed by the gesture design method is verified by tests. Therefore, the AR&DL platform of “AR+DL” establishes a learning platform supported by DL and AR technology. The game decision model is used to describe the process of gesture-based interaction in the game, and the factors affecting the interaction efficiency are reduced, which has certain reference and guidance for VR applications using gesture-based interaction.

为了探索基于深度学习（DL）的虚拟现实（VR）增强现实（AR）移动平台和游戏决策模型的设计，研究了基于Leap Motion的基于手势的VR游戏交互。基于手势的交互特征，建立了VR游戏中的一组通用手势交互规则。同时，根据虚拟现实的理论基础和特点，设计了一套虚拟现实手势交互的通用模型，研究了影响虚拟现实手势交互效率的因素，并设计了合理的交互反馈。通过使用计算机视觉和图像处理技术，基于手势的交互可以收集自然手势，提取手势特征，识别手势指示并响应用户需求。也，它可以从VR中基于手势的交互中提取基本手势，分析VR游戏中手势和基于手势的交互的基本特征，并通过数学矢量和集合来描述手势特征。研究结果表明，手势特征设计方法在游戏中的应用可以分析影响交互效率的因素。另外，通过手势设计方法设计的基于手势的交互的可用性已通过测试验证。因此，“ AR + DL”的AR＆DL平台建立了由DL和AR技术支持的学习平台。该游戏决策模型用于描述游戏中基于手势的交互过程，减少了影响交互效率的因素，对于使用基于手势的交互的VR应用程序具有一定的参考和指导。