In supermarkets or grocery, a shopping cart is a necessary tool for shopping. In this paper, we have developed an intelligent shopping support robot that can carry a shopping cart while following its owners and provide the shopping support by observing the customer’s head orientation, body orientation and recognizing different shopping behaviors. Recognizing shopping behavior or the intensity of such action is important for understanding the best way to support the customer without disturbing him or her. This system also liberates elderly and disabled people from the burden of pushing shopping carts, because our proposed shopping cart is essentially a type of autonomous mobile robots that recognizes its owner and following him or her. The proposed system discretizes the head and body orientation of customer into 8 directions to estimate whether the customer is looking or turning towards a merchandise shelf. From the robot’s video stream, a DNN-based human pose estimator called OpenPose is used to extract the skeleton of 18 joints for each detected body. Using this extracted body joints information, we built a dataset and developed a novel Gated Recurrent Neural Network (GRU) topology to classify different actions that are typically performed in front of shelves: reach to shelf, retract from shelf, hand in shelf, inspect product, inspect shelf. Our GRU network model takes series of 32 frames skeleton data then gives the prediction. Using cross-validation tests, our model achieves an overall accuracy of 82%, which is a significant result. Finally, from the customer’s head orientation, body orientation and shopping behavior recognition we develop a complete system for our shopping support robot.

中文翻译：

---

一个智能的购物支持机器人：使用GRU网络从2D骨架数据了解购物行为

在超市或杂货店，购物车是购物的必备工具。在本文中，我们开发了一种智能购物支持机器人，该机器人可以在跟随其所有者的同时携带购物车，并通过观察客户的头部朝向，身体朝向和识别不同的购物行为来提供购物支持。认识购物行为或这种行为的强度对于理解支持客户而不打扰客户的最佳方法很重要。该系统还使老年人和残疾人摆脱了购物车的负担，因为我们提出的购物车本质上是一种自动移动机器人，可以识别其所有者并跟随他或她。所提出的系统将顾客的头部和身体朝向离散为8个方向，以估计顾客是否正在寻找商品或转向商品货架。从机器人的视频流中，使用基于DNN的人体姿势估计器OpenPose来为每个检测到的身体提取18个关节的骨骼。利用提取的人体关节信息，我们建立了一个数据集并开发了一种新颖的门控递归神经网络（GRU）拓扑结构，以对通常在货架前执行的不同操作进行分类：到达货架，从货架上撤回，手放在货架上，检查产品，检查架子。我们的GRU网络模型采用一系列32帧骨架数据，然后进行预测。通过使用交叉验证测试，我们的模型获得了82％的整体准确度，这是一个显着的结果。最后，