A Contextual Recurrent Collaborative Filtering framework for modelling sequences of venue checkins

Highlights

• We propose the CRCF - the Contextual Recurrent Collaborative Filtering framework - which leverages users’ preferred context and the contextual information associated with the users’ sequence of checkins to model the users’ short-term preferences using a GRU-based RNN component.

• CRCF integrates the state-of-the-art Contextual Recurrent Architecture (CARA) to effectively capture the users’ short-term preferences from their sequence of checkins by incorporating the contextual information associated with their successive checkins.

• We show that high quality venue recommendations for both normal and cold-start users can be generated by the proposed CRCF model.

• We show that sequential geo-based negative sampling approach can improve the effectiveness and robustness of a state-of-the-art neural-network based venue recommendation approach.

Abstract

Context-Aware Venue Recommendation (CAVR) systems aim to effectively generate a ranked list of interesting venues users should visit based on their historical feedback (e.g. checkins) and context (e.g. the time of the day or the user’s current location). Such systems are increasingly deployed by Location-based Social Networks (LBSNs) such as Foursquare and Yelp to enhance the satisfaction of the users. Matrix Factorisation (MF) is a popular Collaborative Filtering (CF) technique that can suggest relevant venues to users based on an assumption that similar users are likely to visit similar venues. In recent years, deep neural networks have been successfully applied to recommendation systems. Indeed, various approaches have been previously proposed in the literature to enhance the effectiveness of MF-based approaches by exploiting Recurrent Neural Networks (RNN) models to capture the sequential properties of observed checkins. Moreover, recently, several RNN architectures have been proposed to incorporate contextual information associated with the users’ sequence of checkins (for instance, the time interval or the geographical distance between two successive checkins) to effectively capture such short-term preferences of users. In this work, we propose a Contextual Recurrent Collaborative Filtering framework (CRCF) that leverages the users’ preferred context and the contextual information associated with the users’ sequence of checkins in order to model the users’ short-term preferences for CAVR. In particular, the CRCF framework is built upon two state-of-the-art approaches: namely Deep Recurrent Collaborative Filtering framework (DRCF) and Contextual Attention Recurrent Architecture (CARA). Thorough experiments on three large checkin and rating datasets from commercial LBSNs demonstrate the effectiveness and robustness of our proposed CRCF framework by significantly outperforming various state-of-the-art matrix factorisation approaches. In particular, the CRCF framework significantly improves NDCG@10 by 5–20% over the state-of-the-art DRCF framework (Manotumruksa, Macdonald, and Ounis, 2017a) and the CARA architecture (Manotumruksa, Macdonald, and Ounis, 2018) across the three datasets. Furthermore, the CRCF framework is less significantly risky than both the DRCF framework and the CARA architecture across the three datasets.

Introduction

Users in Location-Based Social Networks (LBSNs), such as Yelp and Foursquare, tend to search for interesting venues such as restaurants and museums to visit and can share their location with their friends by making checkins at the venues they have visited. This results in large amounts of user checkin data being received by the LBSNs. Such implicit feedback by users also provides rich information about both users and venues, and thus can be leveraged to study the users’ movement in urban cities, as well as to enhance the quality of personalised venue recommendations. Effective Context-Aware Venue Recommendation systems (CAVRs) have become an essential application for LBSNs that allow users to find interesting venues based on their historical checkins and current context (e.g. time of the day, user’s current location as well as their recently visited venues). Matrix Factorisation (MF) (Koren, Bell, & Volinsky, 2009) is a Collaborative Filtering technique that is widely used to generate a personalised ranked list of venues to the users based on their historical checkins. In particular, the MF-based approaches for CAVR typically aim to embed the users’ and venues’ preferences as well as the contextual information about the users within latent factors, which are combined with a dot product operator to estimate the user’s preference for a given venue and context.

Previous studies (Cheng, Yang, Lyu, King, 2013, Rendle, 2012, Tang, Wu, Chen, 2017, Yu, Liu, Wu, Wang, Tan, 2016, Zhang, Dai, Xu, Feng, Wang, Bian, Wang, Liu, 2014) have shown that the sequences of user’s implicit feedback (e.g. sequences of checkins or clicks) play an important role in enhancing the effectiveness of recommendation, across various scenarios. However, traditional MF-based approaches can only capture users’ long-term (static) preferences and not their short-term (dynamic) preferences. Here, dynamic preferences that are captured from the users’ recently visited venues can influence the next venue they may visit (e.g. users may prefer to visit a bar directly after a dinner at a restaurant). In recent years, various approaches have been proposed to leverage Deep Neural Network (DNN) algorithms such as Multi-Layer Perceptron (MLP), Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) for recommendation systems (Cheng, Koc, Harmsen, Shaked, Chandra, Aradhye, Anderson, Corrado, Chai, Ispir, et al., 2016, He, Chua, 2017, He, Liao, Zhang, Nie, Hu, Chua, 2017, Tang, Wu, Chen, 2017, Yu, Liu, Wu, Wang, Tan, 2016). Among various DNN techniques, the RNN models have been widely exploited to extend the MF-based approaches to capture users’ short-term preferences from their sequences of implicit feedback (Beutel, Covington, Jain, Xu, Li, Gatto, Chi, 2018, Smirnova, Vasile, 2017, Tang, Wu, Chen, 2017, Yu, Liu, Wu, Wang, Tan, 2016, Zhu, Li, Liao, Wang, Guan, Liu, Cai, 2017).

A common technique to incorporate RNN models (e.g. Long Short-Term Memory (LSTM) units (Hochreiter & Schmidhuber, 1997) and Gated Recurrent Units (GRU) (Chung, Gulcehre, Cho, & Bengio, 2014)) into MF-based approaches is to feed a sequence of user-venue interactions/checkins into a recurrent model and use the hidden state of the recurrent models to represent the users’ dynamic preferences (Tang, Wu, Chen, 2017, Yu, Liu, Wu, Wang, Tan, 2016, Zhang, Dai, Xu, Feng, Wang, Bian, Wang, Liu, 2014). Next, the user’s preference for a target venue is estimated by calculating the dot product between a latent factor of the user’s dynamic preferences (i.e. the output of the recurrent models) and a latent factor¹ of the target venue. In addition, various approaches have been proposed to extend the RNN models to incorporate the contextual information associated with the sequences of user’s implicit feedback for many recommendation tasks (Beutel, Covington, Jain, Xu, Li, Gatto, Chi, 2018, Jing, Smola, 2017, Liu, Wu, Wang, Tan, 2016, Manotumruksa, Macdonald, Ounis, 2018, Neil, Pfeiffer, Liu, 2016, Smirnova, Vasile, 2017, Twardowski, 2016, Zhu, Li, Liao, Wang, Guan, Liu, Cai, 2017). Recently, (Manotumruksa et al., 2018) proposed a Contextual Attention Recurrent Architecture (CARA) that separately incorporates different types of contextual information associated with the users’ sequence of implicit feedback to model the users’ dynamic preferences for CAVR. The CARA architecture includes two gating mechanisms, namely a Contextual Attention Gate (CAG) and a Time- and Spatial-based Gate (TSG). The CAG controls the influence of context and the previous visited venues, while TSG controls the influence of the hidden state of the previous RNN unit, based on the time interval and the geographical distance between two successive checkins.

Similar to the RNN models proposed in the previous literature (e.g. Beutel, Covington, Jain, Xu, Li, Gatto, Chi, 2018, Smirnova, Vasile, 2017, Zhu, Li, Liao, Wang, Guan, Liu, Cai, 2017, the CARA architecture still relies on a dot product of latent factors of users and items to capture the users’ dynamic preferences in a Collaborative Filtering manner. However, previous works (He, Chua, 2017, Manotumruksa, Macdonald, Ounis, 2017a) have shown that the dot product of latent factors may not be sufficient to capture the complex structures of the user-item interactions (He et al., 2017). Recently, (Manotumruksa et al., 2017a) proposed a Deep Recurrent Collaborative Filtering framework (DRCF) for venue recommendation that leverages the MLP and RNN models to learn the complex structures of the users’ sequences of checkins by replacing the dot product with a neural architecture that can learn an arbitrary function from the sequences of user’ checkins. However, the DRCF framework still relies on the traditional RNN models that are not sufficiently flexible to incorporate the user’s preferred context as well as the contextual information associated with the user’s sequences of checkins.

Both the CARA architecture and the DRCF framework leverage the sequence of user’ implicit feedback (i.e. sequences of checkins) to capture the users’ dynamic preferences. A common challenge that arises when obtaining implicit feedback by observing checkins is that only positive feedback can be observed, and MF-based approaches trained on only positive feedback are likely to be biased to those positive instances. To address this challenge, various negative sampling approaches have been proposed (He, Liao, Zhang, Nie, Hu, Chua, 2017, Manotumruksa, Macdonald, Ounis, 2017a, Manotumruksa, Macdonald, Ounis, 2017b, Rendle, Freudenthaler, Gantner, Schmidt-Thieme, 2009, Yuan, Guo, Jose, Chen, Yu, 2016). For example, the BPR negative sampling approach proposed by Rendle et al. (2009) uniformly and randomly selects venues that the users have not visited as negative instances. Recently, Manotumruksa et al. (2017a) proposed a sequence-based (dynamic) negative sampling approach that takes the sequential properties of checkins and the geographical location of venues into account to enhance the effectiveness of venue recommendation, as well as to alleviate the cold-start user problem. In this article, we aim to address a gap between two state-of-the-art factorisation- and RNN-based approaches (namely the DRCF framework and the CARA architecture) to capture the users’ dynamic preferences when making context-aware venue recommendations, and thereby demonstrate that DRCF and CARA can be effectively combined for this task. Overall, our contributions are summarised below:

• We propose the Contextual Recurrent Collaborative Filtering framework (CRCF), an extension of the DRCF framework (Manotumruksa et al., 2017a), which incorporates both the users’ preferred context and the contextual information associated with the sequence of checkins to effectively capture the users’ dynamic preferences for CAVR. Indeed, the original DRCF framework cannot incorporate the contextual information when generating venue recommendations. Moreover, we propose to integrate the state-of-the-art Contextual Attention Recurrent Architecture (CARA) (Manotumruksa et al., 2018) into our proposed CRCF framework to effectively capture the users’ dynamic preferences.

• We propose to apply a novel sequence-based (dynamic) negative sampling approach proposed by Manotumruksa et al. (2017a) that takes the sequential properties of checkins as well as the geographical location of venues into account to enhance the effectiveness of our CRCF framework. This is proposed in order to alleviate the cold-start user problem.

• We conduct thorough and comprehensive experiments on 3 large-scale real-world datasets, from Brightkite, Foursquare and Yelp, to demonstrate the effectiveness of our proposed CRCF framework for CAVR by comparing it with state-of-the-art venue recommendation approaches. Moreover, we investigate the robustness of the CRCF framework by leveraging risk analyses techniques proposed by Wang, Bennett, and Collins-Thompson (2012) and Dinçer, Macdonald, and Ounis (2014).

The experimental results presented in Section 6 demonstrate that our proposed CRCF framework consistently and significantly outperforms various state-of-the-art venue recommendation approaches in terms of effectiveness and robustness. In particular, our experimental findings are as follows:

• The contextual information associated with the sequences of the users’ checkins (e.g. the time interval and distance between two successive checkins) is important in enhancing the quality of context-aware venue recommendation. Our proposed CRCF framework, which leverages the contextual information can significantly outperform both the state-of-the-art DRCF framework and the CARA architecture on three large datasets.

• Leveraging the sequential order of users’ checkins as well as the geographical information of venues can enhance both the effectiveness and robustness of the CRCF framework. In particular, our experimental results show that the dynamic geo-based negative sampling approach, which takes into account both the sequential order of users checkins and the geographical information of venues, can significantly improve the effectiveness and robustness of various approaches (i.e. the DRCF and CRCF frameworks and the CARA architecture) as well as alleviate the cold-start problem.

• Throughout our comprehensive robustness analysis experiments, we observe that our proposed CRCF framework is significantly less risky, and is less likely to generate poor venue suggestions to the users across the three used datasets, compared to the DRCF framework and the CARA architecture. Moreover, the CRCF framework is more robust than various state-of-the-art venue recommendation approaches (i.e. less likely to generate worse venue suggestions compared to a traditional CF baseline such as BPR).

This article is structured as follows: Section 2 provides the background literature on CAVR, recent trends in applying Deep Neural Networks to recommendation systems as well as various existing extensions of RNN models; Section 3 provides a brief description of the DRCF framework. Section 4 details how to extend the DRCF framework for the CAVR task and also how to integrate the CARA architecture into the resulting CRCF framework; Experimental setup and results are provided in Sections 5 & 6, respectively. Concluding remarks follow in Section 7.