Keita Higuchi
Skip Abstract Section
Abstract
We propose two online-learning algorithms for modeling the personal preferences of users of interactive systems. The proposed algorithms leverage user feedback to estimate user behavior and provide personalized adaptive recommendation for supporting context-dependent decision-making. We formulate preference modeling as online prediction algorithms over a set of learned policies, i.e., policies generated via supervised learning with interaction and context data collected from previous users. The algorithms then adapt to a target user by learning the policy that best predicts that user’s behavior and preferences. We also generalize the proposed algorithms for a more challenging learning case in which they are restricted to a limited number of trained policies at each timestep, i.e., for mobile settings with limited resources. While the proposed algorithms are kept general for use in a variety of domains, we developed an image-filter-selection application. We used this application to demonstrate how the proposed algorithms can quickly learn to match the current user’s selections. Based on these evaluations, we show that (1) the proposed algorithms exhibit better prediction accuracy compared to traditional supervised learning and bandit algorithms, (2) our algorithms are robust under challenging limited prediction settings in which a smaller number of expert policies is assumed. Finally, we conducted a user study to demonstrate how presenting users with the prediction results of our algorithms significantly improves the efficiency of the overall interaction experience.
- Charu C. Aggarwal. 2016. Recommender Systems: The Textbook (1st ed.). Springer Publishing Company, Incorporated. Google Scholar
Digital Library
- Eugene Agichtein, Eric Brill, Susan Dumais, and Robert Ragno. 2006. Learning user interaction models for predicting web search result preferences. In Proceedings of the 29th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’06). ACM, New York, NY, 3--10. DOI:https://doi.org/10.1145/1148170.1148175 Google ScholarDigital Library
- Christopher J. Anderson. 2003. The psychology of doing nothing: Forms of decision avoidance result from reason and emotion. Psychol. Bull. 129, 1 (2003), 139.Google ScholarCross Ref
- Peter Auer. 2003. Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3 (Mar. 2003), 397--422. Retrieved from http://dl.acm.org/citation.cfm?id=944919.944941. Google ScholarDigital Library
- Linas Baltrunas, Marius Kaminskas, Bernd Ludwig, Omar Moling, Francesco Ricci, Aykan Aydin, Karl-Heinz Lüke, and Roland Schwaiger. 2011. InCarMusic: Context-aware music recommendations in a car. In E-Commerce and Web Technologies, Christian Huemer and Thomas Setzer (Eds.). Springer Berlin, 89--100.Google Scholar
- Björn Brodén, Mikael Hammar, Bengt J. Nilsson, and Dimitris Paraschakis. 2018. Ensemble recommendations via Thompson sampling: An experimental study within e-commerce. In Proceedings of the 23rd International Conference on Intelligent User Interfaces (IUI’18). ACM, New York, NY, 19--29. DOI:https://doi.org/10.1145/3172944.3172967 Google ScholarDigital Library
- Nicolò Cesa-Bianchi, Yoav Freund, David Haussler, David P. Helmbold, Robert E. Schapire, and Manfred K. Warmuth. 1997. How to use expert advice. J. ACM 44, 3 (May 1997), 427--485. DOI:https://doi.org/10.1145/258128.258179 Google ScholarDigital Library
- Nicolo Cesa-Bianchi and Gabor Lugosi. 2006. Prediction, Learning, and Games. Cambridge University Press, New York, NY. Google ScholarDigital Library
- Olivier Chapelle and Lihong Li. 2011. An empirical evaluation of Thompson sampling. In Advances in Neural Information Processing Systems 24, J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 2249--2257. Retrieved from http://papers.nips.cc/paper/4321-an-empirical-evaluation-of-thompson-sampling.pdf. Google ScholarDigital Library
- Xiangmin Fan, Youming Liu, Nan Cao, Jason Hong, and Jingtao Wang. 2015. MindMiner: A mixed-initiative interface for interactive distance metric learning. In Human-Computer Interaction -- INTERACT 2015, Julio Abascal, Simone Barbosa, Mirko Fetter, Tom Gross, Philippe Palanque, and Marco Winckler (Eds.). Springer International Publishing, Cham, 611--628.Google Scholar
- Leah Findlater and Jacob Wobbrock. 2012. Personalized input: Improving ten-finger touchscreen typing through automatic adaptation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 815--824. DOI:https://doi.org/10.1145/2207676.2208520 Google ScholarDigital Library
- James Fogarty, Desney Tan, Ashish Kapoor, and Simon Winder. 2008. CueFlik: Interactive concept learning in image search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’08). 29--38. DOI:https://doi.org/10.1145/1357054.1357061 Google ScholarDigital Library
- Johannes Frnkranz and Eyke Hllermeier. 2010. Preference Learning (1st ed.). Springer-Verlag, Berlin. Google ScholarDigital Library
- Krzysztof Z. Gajos, Daniel S. Weld, and Jacob O. Wobbrock. 2010. Automatically generating personalized user interfaces with Supple. Artif. Intell. 174, 12 (2010), 910--950. DOI:https://doi.org/10.1016/j.artint.2010.05.005 Google ScholarDigital Library
- Aurélien Garivier and Olivier Cappé. 2011. The KL-UCB algorithm for bounded stochastic bandits and beyond. In Proceedings of the 24th Conference on Learning Theory. 359--376.Google Scholar
- Satyen Kale. 2014. Multiarmed bandits with limited expert advice. In Proceedings of the Conference on Learning Theory. 107--122.Google Scholar
- Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2014. Crowd-powered parameter analysis for visual design exploration. In Proceedings of the 27th ACM Symposium on User Interface Software and Technology (UIST’14). ACM, New York, NY, 65--74. DOI:https://doi.org/10.1145/2642918.2647386 Google ScholarDigital Library
- Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2016. SelPh: Progressive learning and support of manual photo color enhancement. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 2520--2532. DOI:https://doi.org/10.1145/2858036.2858111 Google ScholarDigital Library
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 1097--1105. Retrieved from http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf. Google ScholarDigital Library
- Seok Kee Lee, Yoon Ho Cho, and Soung Hie Kim. 2010. Collaborative filtering with ordinal scale-based implicit ratings for mobile music recommendations. Inf. Sci. 180, 11 (2010), 2142--2155. DOI:https://doi.org/10.1016/j.ins.2010.02.004 Google ScholarDigital Library
- Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). ACM, New York, NY, 661--670. DOI:https://doi.org/10.1145/1772690.1772758 Google ScholarDigital Library
- G. Linden, B. Smith, and J. York. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 7, 1 (Jan 2003), 76--80. DOI:https://doi.org/10.1109/MIC.2003.1167344 Google ScholarDigital Library
- Jiming Liu, Chi Kuen Wong, and Ka Keung Hui. 2003. An adaptive user interface based on personalized learning. IEEE Intell. Syst. 18, 2 (Mar. 2003), 52--57. DOI:https://doi.org/10.1109/MIS.2003.1193657 Google ScholarDigital Library
- Abhinav Mehrotra, Robert Hendley, and Mirco Musolesi. 2016. PrefMiner: Mining user’s preferences for intelligent mobile notification management. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’16). ACM, New York, NY, 1223--1234. DOI:https://doi.org/10.1145/2971648.2971747 Google ScholarDigital Library
- Pablo Messina, Vicente Dominguez, Denis Parra, Christoph Trattner, and Alvaro Soto. 2019. Content-based artwork recommendation: Integrating painting metadata with neural and manually-engineered visual features. User Model. User-adapt. Interact. 29, 2 (01 Apr. 2019), 251--290. DOI:https://doi.org/10.1007/s11257-018-9206-9 Google ScholarDigital Library
- Lijing Qin, Shouyuan Chen, and Xiaoyan Zhu. [n.d.]. Contextual Combinatorial Bandit and Its Application on Diversified Online Recommendation. 461--469. DOI:https://doi.org/10.1137/1.9781611973440.53Google Scholar
- Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web (WWW’01). 285--295. DOI:https://doi.org/10.1145/371920.372071 Google ScholarDigital Library
- Yevgeny Seldin, Peter Bartlett, Koby Crammer, and Yasin Abbasi-Yadkori. 2014. Prediction with limited advice and multiarmed bandits with paid observations. In Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32 (ICML’14). JMLR.org, I--280--I--287. Retrieved from http://dl.acm.org/citation.cfm?id=3044805.3044838. Google ScholarDigital Library
- Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014).Google Scholar
- William R. Thompson. 1933. On the likelihood that one unknown probablity exceeds another in view of the evidence of two samples. Biometrika 25, 3--4 (12 1933), 285--294. DOI:https://doi.org/10.1093/biomet/25.3-4.285Google ScholarCross Ref
- John Tierney. 2011. Do you suffer from decision fatigue. The New York Times (2011).Google Scholar
- Aaron van den Oord, Sander Dieleman, and Benjamin Schrauwen. 2013. Deep content-based music recommendation. In Advances in Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 2643--2651. Retrieved from http://papers.nips.cc/paper/5004-deep-content-based-music-recommendation.pdf. Google ScholarDigital Library
- Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 (NIPS’14). The MIT Press, Cambridge, MA, 3320--3328. Retrieved from http://dl.acm.org/citation.cfm?id=2969033.2969197. Google ScholarDigital Library
- Donggyu Yun, Alexandre Proutiere, Sumyeong Ahn, Jinwoo Shin, and Yung Yi. 2018. Multi-armed bandit with additional observations. Proc. ACM Meas. Anal. Comput. Syst. 2, 1 (Apr. 2018). DOI:https://doi.org/10.1145/3179416 Google ScholarDigital Library
- Hengshu Zhu, Enhong Chen, Hui Xiong, Kuifei Yu, Huanhuan Cao, and Jilei Tian. 2014. Mining mobile user preferences for personalized context-aware recommendation. ACM Trans. Intell. Syst. Technol. 5, 4 (Dec. 2014). DOI:https://doi.org/10.1145/2532515 Google ScholarDigital Library
Index Terms
- Learning Context-dependent Personal Preferences for Adaptive Recommendation
Recommendations
Adaptive personalized recommendation based on adaptive learning
Collaborative filtering has been widely applied in many fields in recent years due to the increase in web-based activities such as e-commerce and online content distribution. Current collaborative filtering techniques such as correlation-based, SVD-...
Context-aware adaptive personalised recommendation: a meta-hybrid
Recommenders take place on a wide scale of e-commerce systems, reducing the problem of information overload. The most common approach is to choose a recommender used by the system to make predictions. However, users vary from each other; thus, a one-fits-...
Improving Cold Start Recommendation by Mapping Feature-Based Preferences to Item Comparisons
UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and PersonalizationMany Recommender Systems (RSs) rely on user preference data in the form of ratings or likes for items. Previous research has shown that item comparisons can also be effectively used to model user preferences and build RS. However, users often express ...
Comments