Inference of Other’s Minds with Limited Information in Evolutionary Robotics

Abstract

Theory of mind (ToM) is the ability to understand others’ mental states (e.g., intentions). Studies on human ToM show that the way we understand others’ mental states is very efficient, in the sense that observing only some portion of others’ behaviors can lead to successful performance. Recently, ToM has gained interest in robotics to build robots that can engage in complex social interactions. Although it has been shown that robots can infer others’ internal states, there has been limited focus on the data utilization of ToM mechanisms in robots. Here we show that robots can infer others’ intentions based on limited information by selectively and flexibly using behavioral cues similar to humans. To test such data utilization, we impaired certain parts of an actor robot’s behavioral information given to the observer, and compared the observer’s performance under each impairment condition. We found that although the observer’s performance was not perfect compared to when all information was available, it could infer the actor’s mind to a degree if the goal-relevant information was intact. These results demonstrate that, similar to humans, robots can learn to infer others’ mental states with limited information.

Appendices

Appendix A: Actor Learning

Actor’s neural network is defined with the real-valued parameters. It is assumed that observer has no direct access to the parameters but attempts to infer them from the behaviors (trajectories) of the actor. The actor’s NN evolved to move towards a light source placed in the environment (the light is placed at (L_X, L_Y)). The evolution aims at finding the parameters (weights) in the NN that controls the robot behavior. Because these parameters are real-valued, there are a very large number of NN candidates. An evolutionary algorithm, inspired by natural evolution, guides the search for the NN parameters.

In this paper, we adopt an evolutionary strategy (ES) [45], which has been used for many engineering problems (for example, game strategy and analog circuit evolution [46,47,48]). The ES is relatively simpler than other evolutionary methods but it has been successful in optimizing real-valued parameters for engineering problems. For example, it successfully optimized 1741 weights of neural networks which played checkers (better than 99.61% of the playing population of zone.com players) [48].

In ES, each solution initializes the neural network’s weights and corresponding mutation step-size. The evolutionary search optimizes both the mutation strength and the weights of NNs together. Only those individuals with high fitness value get a chance of selection. It is a deterministic process in which only the best half from the pool of parents and offspring survive to the next generation. The selection technique is called “truncation” or “breeding” selection [45].

The details of the ES are as follows. Initially, P NNs (parents) are generated randomly (P is the population size). Weights (including the bias weights) are selected from a uniform distribution with a range of − 0.2 to 0.2. Each weight has a corresponding mutation step-size initialized at 0.05. Each parent NN generates one offspring through a mutation yielding 2 × P neural networks (parents + offspring). The mutation operator is defined as follows (it slightly changes the current weight $w_{i} (j)$ to new one $w^{\prime}_{i} (j)$ to produce the offspring):

$$ \begin{aligned} \, \sigma_{i}^{\prime } (j) & = \sigma_{i} (j)\exp \left( {\tau \times N_{j} (0,1)} \right) \, \\ \, w_{i}^{\prime } (j) & = w_{i} (j) + \sigma_{i}^{\prime } (j)N_{j} (0,1) \\ \end{aligned} $$

(10)

where N_w is the number of weights, $\tau$ is the learning parameter, $w_{i} (j)$ is the jth weight of the ith neural network in the population, $\sigma_{i} (j)$ is the corresponding mutation-step size for $w_{i} (j)$ and N_j(0,1) is a standard Gaussian random variable re-sampled for every j. The parameter is defined as follows.

$$ \tau = 1/\sqrt {2\sqrt {N_{W} } } $$

(11)

The parameter in Eq. (11) is chosen using theoretical and empirical evidence [49].

From a pool of parents and offspring, only half survive to the next generation based on fitness. Because the goal of this evolution was to reach the light, fitness was measured by the Euclidean distances between the robot and the light source at each time point during the navigation.

$$ Fitness_{i} = \frac{1}{{\sum\nolimits_{j = 1}^{MAX\_STEPS} {\sqrt {(L\_X - R\_X(j))^{2} + (L\_Y - R\_Y(j))^{2} } } }} $$

(12)

It sorts the 2 × P candidate NNs (parents + offspring) based on the fitness. Only half of them survive to be parents in the next generation.

Appendix B: Robot’s Details

• Body (Morphology) The robot is like a tricycle which has a big main body and three wheels (one front wheel and two rear wheels) (Fig. 3a). The radius of the sphere is 1 m. The wheel’s radius and width are 0.5 and 0.3 meters, respectively. The density of the robot is 5 kg/m³. It is modified based on the sample tricycle (with a rectangular body) from the PhysX simulator.

• Sensors The robots have two light sensors. Sensors are located on the front side of the upper hemisphere of the robot’s body and they detect light levels around the body. The sensors are located at + π/4 and − π/4 positions. The light levels are measured using the following equation (r: Euclidean distance between the sensor and the light source, θ: the angle between the sensor and the light source)

  $$ sensor\_value = \frac{1000}{{r^{2} }}\cos (\theta ) $$

  (13)

• Actuators At each time step (per 1/60 s), the simulator sets the angle (− 1/3π to 1/3π) of the front wheel and the speed of the rear wheels based on the outputs from the controller. The maximum speed of the robot is 4 m/s.

• Controller The robots received two (right and left) light sensor values which are sent to the NN, producing two real-values for the direction and speed of the wheels (Fig. 3b). The model has three hidden neurons and two output neurons. The number of weights is 17, consisting of 12 connection weights and 5 biases. Because each neuron has one bias, the number of biases is the same to that of neurons in the neural network. A hyperbolic tangent is used as a sigmoid function. The actor learned to move towards a light source by evolving an “innate” NN through its interaction with the environment. In this way, the actor is able to follow the light source using its unique neural controller (see Fig. 10).

  Fig. 10

  

  The behavior and neural topology of the actor. a The figure shows the trajectories of the neural network (the black cross is the light source and the circle represents the starting positions of the robot. The robot’s initial angle is set as 0 degree.) b The actor robot’s neural network model evolved to reach the light. The figure shows the weights of the actor’s NN evolved