Language model based interactive estimation of distribution algorithm

Highlights

• The presented IEDA employs a language model to encode candidate searched items.

• The language model introduces social intelligence and reduces information loss.

• The IEDA adopts Dirichlet-Multinomial distribution as its probabilistic model.

• The probabilistic model is updated with Bayesian learning to track variable.

• Then, a faster personalized search can be expected.

Abstract

It is very hard, if not impossible to use analytical objective functions for optimization of personalized search due to the difficulties in mathematically describing qualitative problems. To solve such optimization problems, interactive evolutionary algorithms, which can make use of human preferences, are highly desirable. However, due to the lack of effective encoding methods, interactive evolutionary algorithms have been limited to numerically encoded optimization problems. In practice, however, linguistic terms (words) are the most natural expression of human preferences, and they are also commonly used to describe items in personalized search or E-commerce; therefore, language models better suit encoding, and the optimization of personalized search is converted into a dynamic document matching problem. To optimize word-described personalized search, we propose a novel interactive estimation of distribution algorithm. This algorithm combines a language model-based encoding approach, a Dirichlet-Multinomial compound distribution-based preference expression, and a Bayesian inference mechanism. The proposed algorithm is applied to two personalized search cases to demonstrate the capability of the algorithm in ensuring a more efficient and accurate search with less user fatigue.

Introduction

Objective functions of personalized optimization problems, e.g. product design, personalized search, and information retrieval, are impossible to precisely define using mathematical expressions since they are highly dependent on user preferences. Although evolutionary computation (EC) has been proven to be a powerful tool for solving complex optimization [1], [2], [3] problems, the requirement of precise mathematical definitions cannot be fulfilled in personalized search. In such scenarios, interactive evolutionary computation (IEC) [4], [5] is more feasible and efficient as it involves a human user in the evaluation process; they have been developed and successfully applied to various practical problems, such as product design, web page layout design, and anti-collision design of vehicles [5], [6], [7], [8]. This paper considers the optimization problems that occur in, for example, the following scenario: A person is searching the web for a particular movie, beginning the search with a query of a few words. The search engine presents a few movie candidates. The user clicks on some of the candidates and saves some of them. Based on the user’s actions, the search engine presents some new candidates. This process continues until the user is satisfied with the result. The purpose of this proposed novel method is to speed up the search process and reduce the need for user interference.

IEC requires human evaluations, which can inevitably cause user fatigue given a complex problem. The restriction on population size and evolutionary generation prevents the use of IEC in tackling a wide range of problems [5]. Accordingly, much more attention has been paid to alleviating user fatigue and improving explorations from the following three aspects [9]: (1) the design of friendly human–computer interfaces or novel evaluation modes to reduce the user burden, e.g. evaluating individuals with discrete, fuzzy, or interval numbers [9], [10], [11]; (2) the use of a preference surrogate, with a small number of evaluated individuals and then apply it to help with the assessment. With surrogates, IEC can upscale the population size and generation as conventional EC approaches [12], [13], which greatly improves the explorations of IEC. (3) the use of knowledge from evolution to modify evolutionary operators to accelerate search and reduce fatigue [14], [15]. These studies have greatly enhanced IEC. Although the above-mentioned studies have greatly enhanced IEC , the application of those methods in resolving preference-related complex problems remains a challenging task. Particularly in word-described ones such as online personalized search in E-commerce.

The main reason is that the information covered by the numerical representations for these word-described problems is insufficient to model the preference-related objectives, leading to inefficient or even wrong searches. Sun et al. [9], [16] used a limited number of numerical values to encode these word-described items in the framework of an interactive genetic algorithm for the personalized search, which made the traditional evolutionary operators easier to implement. Wang et al. [17] modeled TV programs with five attributes in their experiments when studying preference recommendations for personalized search.

These studies are easier to understand and implement; however, such numerical encoding loses a considerable amount of implied semantic information contained in the words. In addition, IEC depends on the evaluations assigned by the users, who have got used to thinking and evaluating with words instead of numbers. The gap between the users’ evaluations and numerical encoding results in a need for additional human–computer interactions and inevitably causes more user fatigue. Accordingly, designing a non-numerical encoding method which minimizes loss of semantic information and developing corresponding evolutionary operators becomes essential as this will help to enhance the performance of IEC in solving more practical and complex problems. Furthermore, user preferences or decisions will be influenced greatly by other user comments, and social or group comments should be integrated with IEC so that the current user is able to precisely evaluate the searched solutions. Motivated by the above, we focus on developing an enhanced IEC by integrating social comments, designing a novel encoding and corresponding evolutionary operators for solving problems described with words in the personalized search.

Applying IEC, short for Interactive Evolutionary Computation, to the word/document-described optimization problems relies on establishing a bridge between the textual phenotype evaluated by the user and the numerical genotype operated by EC. The language model Doc2Vec [18], [19] is a good choice to convert textual phenotypes into numerical vectors by preserving most of the semantic relationships among the words. Therefore, we employ this model to represent the genotype of a document, i.e., a searched item, including the word description and social comments. Clearly, both are naturally combined in the Doc2Vec based representation. Given this, a new IEC must be developed to gain the advantages both from itself and the model, i.e., Doc2Vec-assisted initialization and interactive evolutionary operators.

This work develops a language model based interactive estimation of distribution algorithm (LMIEDA) to perform the evolutionary optimization in personalized search. In LMIEDA, the Doc2Vec is applied to convert the word/document-described problem into a dynamic document matching one by encoding the word frequency as individuals. A preference function is approximately constructed based on user interactions and is used to estimate the individual’s fitness. Based on the fitness, the Dirichlet-Multinomial compound distribution and a Bayesian inference involving the Dirichlet-Multinomial compound distribution is designed to track the user’s preference on the variables. With these, the probabilistic model of estimation of distribution algorithm (EDA) and the corresponding sampling are presented. The user’s burden here can be greatly reduced since our algorithm is able to estimate the fitness of all individuals without the user.

The main contributions of this study are as follows. (1) To the best of our knowledge, language models have not been used in EC/IEC. As an encoding method, it helps to reduce information loss and naturally introduces social intelligence. (2) Since the encoded variables are discrete (with finite states), the Dirichlet-Multinomial compound distribution is adopted as the probabilistic model of IEDA to be compatible with encoded candidates. (3) The probabilistic model is updated with the help of Bayesian learning to directly track variable distribution, and most conventional EDAs employ Bayesian networks to depict the dependencies between variables. (4) The proposed algorithm is applied to some personalized search for books and movies to prove its effectiveness and efficiency.

The remainder of this paper is organized as follows. Section 2 describes the related work on the personalized search assisted with evolutionary algorithms, the estimation of distribution algorithms (EDAs), and the basic concept of Doc2Vec. The details of the proposed algorithm, including the definition of the word-described personalized search, the framework, the critical encoding, preference expression, and the IEDA, are presented in Section 3. Section 4 addresses the application of the proposed algorithm together with the experimental results and analyses. Conclusions are drawn in Section 5.