Competition and cooperation in a community of autonomous agents

Abstract

Agents that perform intelligent tasks interacting with humans in a seamless manner are becoming a reality. In contexts in which interactions among agents repeat over time, they might evolve from a cooperative to a competitive attitude, and vice versa, depending on environmental factors and other contextual circumstances. We provide a framework to model transitions between competition and cooperation in a community of agents. Competition is dealt with through the paradigm of adversarial risk analysis, which provides a disagreement solution; implicitly, we minimize the distance to such solution. Cooperation is handled through a concept of maximal separation from the disagreement solution. Mixtures of both problems are used to refer to in-between behaviour. We illustrate the ideas with several simulations in relation with a group of robots. Our motivation is the constitution of communities of robotic agents that interact among them and with one or more users.

Experimental setup

We describe in some detail the setup for our experiments. The chosen robotic platform has been selected for being a low-cost solution affordable for being at home and schools. This robot has several sensors, including a camera to detect objects or persons within a scene; a microphone used to recognize and understand what the user says through an ASR component; touch sensors to interpret when it has been stroked or attacked; an inclination sensor to know whether it is in vertical position; and several actuators, including servos that allow the robot to move some of its parts, although it mostly uses a TTS system and a simple screen to simulate a mouth when talking.

Our social agents will be dynamically aware of their external context which comprises their environment E as well as the actions performed by a user B during interactions. The evolution of environmental conditions due to the agents’ and user’s actions is assumed to be perceptible through the sensors installed in the agents as described above. The agents’ decisions a will be regulated and planned within the environment, which changes with the user’s actions b, made within an action set ${{\mathcal {B}}}$, leading to an environmental state e within a set ${{\mathcal {E}}}$.

The global loop of the robots covers the stages of: (i)sensing and forecasting; and (ii) decision making.

At time t, the agents step into the first stage. They collect signals from their sensors to interpret the environmental conditions and user actions. The forecasting model is used in expected utility calculations to determine optimal decisions. Once the agents perform their actions, the user responds and the environment evolves. As soon as the agents receive the user responses, they assess the actual consequences of all decisions. Then, the control values are adapted and the environmental states updated, with the time mechanism forwarded in last.

We provide now some details of all the relevant elements. The underlying decision making model uses multi-attribute expected utilities with probabilities based on the ARA framework, adapting ideas from Esteban and Ríos Insua (2014, 2015). We refer to the action sets of the agents and the user, respectively, as ${{\mathcal {A}}}_i =\{a_1,\ldots ,a_{15}\}$ and ${{\mathcal {B}}} = \{b_1,\ldots ,b_{14}\}$. The simulated agents may thus perform 15 actions, divided in four groups labeled attention-seeking, complaining, unresponsive and interactive. They may also detect 14 user actions which we have divided as affective, aggressive, interactive, unresponsive and updating, as reflected in Table 6.

At time t, depending on the actions $a_t$ of the agents, the action $b_t$ of the user and the environmental state $e_t \in {{\mathcal {E}}} = \{e_1,\ldots ,e_r\}$, the agents obtain the multi-attribute consequences $c^i(a_t,b_t,e_t)$, $i = 1,\ldots ,l$. Specifically, we set $l = 5$, being the objectives:

$u_1$:

Being sufficiently charged.

$u_2$:

Remain secure, in relation with the noise, light and temperature conditions surrounding the agent.

$u_3$:

Interact with identified users.

$u_4$:

Having fun with identified users.

$u_5$:

Having the software updated.

The utility function adopts the multi-attribute additive form for the i-th agent

$$\begin{aligned} u(c^1,\ldots ,c^5) = \sum _{k=1}^{5} {\omega _i^k\,u_i^k}, \end{aligned}$$

where $\omega _i^k \ge 0$ and $\sum _{k=1}^{5} {w_i^k} = 1$. $w_i^k$ represents the weight of the i-th agent’s k-th objective and $u_i^k$ represents the corresponding component utility function. We set $w_1> w_2> w_3> w_4 > w_5$ to stress the hierarchical nature of the objectives.

The agents’ beliefs are regulated within the ARA framework, more specifically within the level-1 thinking approach. Given the past history of the agents’ and user’s actions, environmental states and the agents’ potential action $a_t$, each agent forecasts the user’s action and environment state through

$$\begin{aligned} p(e_t,b_t\,|\,a_t,(e_{t-1},a_{t-1},b_{t-1}),(e_{t-2},a_{t-2},b_{t-2})), \end{aligned}$$

(9)

where we limit memory to two periods for computational reasons. We decompose (9) through

$$\begin{aligned} p(e_t\,|\,b_t,e_{t-1},e_{t-2}) \cdot p(b_t\,|\,a_t,b_{t-1},b_{t-2}). \end{aligned}$$

Then, under standard conditions, the agents are designed to choose the action with Maximum Expected Utility (MEU), that is, they will solve

$$\begin{aligned} \psi _t^*= & {} \max _{a_t \in {{\mathcal {A}}}} \psi (a_t) \\ \displaystyle= & {} \iint u(a_t,b_t,e_t)\,p(e_t\,|\,b_t,e_{t-1},e_{t-2}) \\&\qquad \qquad \displaystyle p(b_t\,|\,a_t,b_{t-1},b_{t-2})\,\text {d}b_t\,\text {d}de_t. \end{aligned}$$

Note that, for computational reasons, we just plan one period ahead. The ideas may be easily extended to longer planning periods through dynamic programming. In such case, a more stable class of weights could be obtained as the average of a few of the last utilities attained.