As mobile shopping has gradually become the mainstream shopping mode, recommendation systems are gaining an increasingly wide adoption. Existing recommendation systems are mainly based on explicit and implicit user behaviors. However, these user behaviors may not directly indicate users’ inner feelings, causing erroneous user preference estimation and thus leading to inaccurate recommendations. Inspired by our key observation on the correlation between pupil size and users’ inner feelings, we consider using the change of pupil size when browsing to model users’ preferences, so as to achieve targeted recommendations. To this end, we propose PupilRec as a computer-vision-based recommendation framework involving a mobile terminal and a server side. On the mobile terminal, PupilRec collects users’ pupil size change information through the front camera of smartphones; it then preprocesses the raw pupil size data before transmitting them to the server. On the server side, PupilRec utilizes the Tsfresh package and Random Forest algorithm to figure out the key time-series features directly implying user preferences. PupilRec then trains a neural network to fit a user preference model. Using this model, PupilRec predicts user preference to obtain a user–product matrix and further simplifies it by singular value decomposition. Finally, the real-time recommendation is achieved by a collaborative filtering module that retrieves recommended contents to users smartphones. We prototype PupilRec and conduct both experiments and field studies to comprehensively evaluate the effectiveness of PupilRec by recruiting 67 volunteers. The overall results show that PupilRec can accurately estimate users’ preference and can recommend products users interested in.

中文翻译：

---

PupilRec：利用瞳孔形态在智能手机上进行推荐


随着移动购物逐渐成为主流购物方式，推荐系统的应用也越来越广泛。现有的推荐系统主要基于显式和隐式的用户行为。然而，这些用户行为可能无法直接反映用户的内心感受，从而导致错误的用户偏好估计，从而导致推荐不准确。受到我们对瞳孔大小与用户内心感受之间相关性的重点观察的启发，我们考虑利用浏览时瞳孔大小的变化来模拟用户的偏好，从而实现有针对性的推荐。为此，我们提出 PupilRec 作为涉及移动终端和服务器端的基于计算机视觉的推荐框架。在移动端，PupilRec通过智能手机前置摄像头采集用户瞳孔大小变化信息；然后，它会在将原始瞳孔大小数据传输到服务器之前对其进行预处理。在服务器端，PupilRec利用Tsfresh包和随机森林算法来找出直接暗示用户偏好的关键时间序列特征。 PupilRec 然后训练神经网络以适应用户偏好模型。 PupilRec 使用该模型预测用户偏好以获得用户-产品矩阵，并通过奇异值分解进一步简化它。最后，通过协同过滤模块实现实时推荐，该模块将推荐内容检索到用户的智能手机。我们对 PupilRec 进行了原型设计，并招募了 67 名志愿者进行实验和现场研究，以全面评估 PupilRec 的有效性。总体结果表明，PupilRec能够准确估计用户的偏好，并能够推荐用户感兴趣的产品。