Deep soccer analytics: learning an action-value function for evaluating soccer players

Abstract

Given the large pitch, numerous players, limited player turnovers, and sparse scoring, soccer is arguably the most challenging to analyze of all the major team sports. In this work, we develop a new approach to evaluating all types of soccer actions from play-by-play event data. Our approach utilizes a Deep Reinforcement Learning (DRL) model to learn an action-value Q-function. To our knowledge, this is the first action-value function based on DRL methods for a comprehensive set of soccer actions. Our neural architecture fits continuous game context signals and sequential features within a play with two stacked LSTM towers, one for the home team and one for the away team separately. To validate the model performance, we illustrate both temporal and spatial projections of the learned Q-function, and conduct a calibration experiment to study the data fit under different game contexts. Our novel soccer Goal Impact Metric (GIM) applies values from the learned Q-function, to measure a player’s overall performance by the aggregate impact values of his actions over all the games in a season. To interpret the impact values, a mimic regression tree is built to find the game features that influence the values most. As an application of our GIM metric, we conduct a case study to rank players in the English Football League Championship. Empirical evaluation indicates GIM is a temporally stable metric, and its correlations with standard measures of soccer success are higher than that computed with other state-of-the-art soccer metrics.

Proof of Proposition 1

The data record transitions from a state-action-player triple to another, possibly resulting in a non-zero reward (score or point in the context of sports). We denote the number of times such a transition occurs as

$$\begin{aligned} n_{D}[s,a,{ pl},s',a',{ pl}'] \end{aligned}$$

where the $'$ indicates the successor triple. We freely use this notation for marginal counts as well, for instance

$$\begin{aligned} n_{D}[s',a',{ pl}'] = \sum _{s,a,{ pl}}n_{D}[s,a,{ pl},s',a',{ pl}'] \end{aligned}$$

From the paper, we have the following equations for the Q-value-above-replacement and the GIM metrics:

$$\begin{aligned} { QAAR}^{i}(D)&= \sum _{s,a} n_{D}[s,a,{ pl}' = i] \big ( {{\,\mathrm{{\mathbb {E}}}\,}}_{s',a'}[Q_{{ team}}(s',a'|s,a,{ pl}'=i)] \nonumber \\&\quad - {{\,\mathrm{{\mathbb {E}}}\,}}_{s',a'}[Q_{{ team}}(s',a')|s,a] \big ) \end{aligned}$$

(7)

$$\begin{aligned} GIM^{i}(D)&= \sum _{s,a,s',a'}n[s,a,s',a',{ pl}'=i;D] \cdot \Big [Q_{{ team}}(s',a') \nonumber \\&\quad - {{\,\mathrm{{\mathbb {E}}}\,}}_{s'_{E},a'_{E}}[Q_{{ team}}(s'_{E},a'_{E})|s,a]\Big ] \end{aligned}$$

(8)

Now we have

$$\begin{aligned} GIM^{i}(D){\mathop {=}\limits ^{Eq.2}}&\sum _{s,a} \sum _{s',a'} n_{D}[s,a,s',a',{ pl}' = i] \Big (Q_{{ team}}(s',a')- {{\,\mathrm{{\mathbb {E}}}\,}}_{s'_{E},a'_{E}}[Q_{{ team}}(s'_{E},a'_{E})|s,a]\Big ) \nonumber \\ =&\sum _{s,a} n_{D}[s,a,{ pl}' = i] \sum _{s',a'} \frac{n_{D}[s,a,s',a',{ pl}' = i]}{n_{D}[s,a,{ pl}' = i]} Q_{{ team}}(s',a') \nonumber \\&- \sum _{s,a} n_{D}[s,a,{ pl}' = i] {{\,\mathrm{{\mathbb {E}}}\,}}_{s'_{E},a'_{E}}[Q_{{ team}}(s'_{E},a'_{E})|s,a] \end{aligned}$$

(9)

$$\begin{aligned} =&\sum _{s,a} n_{D}[s,a,{ pl}' = i] E[Q_{{ team}}(s',a'|s,a,{ pl}'=i)] \end{aligned}$$

(10)

$$\begin{aligned}&- \sum _{s,a} n_{D}[s,a,{ pl}' = i] {{\,\mathrm{{\mathbb {E}}}\,}}_{s'_{E},a'_{E}}[Q_{{ team}}(s'_{E},a'_{E})|s,a] \nonumber \\ =&\sum _{s,a} n_{D}[s,a,{ pl}' = i] \big ( {{\,\mathrm{{\mathbb {E}}}\,}}_{s'_{E},a'_{E}}[Q_{{ team}}(s'_{E},a'_{E}|s,a,{ pl}'=i)] \nonumber \\&\quad - {{\,\mathrm{{\mathbb {E}}}\,}}_{s'_{E},a'_{E}}[Q_{{ team}}(s'_{E},a'_{E})|s,a] \big ) \nonumber \\ {\mathop {=}\limits ^{Eq.1}}&{ QAAR}^{i}(D) \end{aligned}$$

(11)

Step (9) holds because the expectation $E[Q_{{ team}}(s',a'|s,a)]$ depends only on $s,a$, not on $s',a'$. Line (10) uses the empirical estimate of the expected Q-value $Q_{{ team}}(s',a')]$ given that player i acts next, computed from the maximum likelihood estimates of the transition probabilities:

$$\begin{aligned} {\hat{\sigma }}(s',a'|s,a,{ pl}' = i) = \frac{n_{D}[s,a,s',a',{ pl}' = i]}{n_{D}[s,a,{ pl}' = i]} \end{aligned}$$

The final conclusion (11) applies Equation (7).