Runtime analysis of evolutionary algorithms via symmetry arguments

https://doi.org/10.1016/j.ipl.2020.106064Get rights and content

Highlights

  • An exponential lower bound for the runtime of a basic crossover-based evolutionary algorithms.

  • A simple and natural proof method that could be helpful also for other lower bound problems.

  • A way to analyze evolutionary algorithms with non-random initializations.

Abstract

We use an elementary argument building on group actions to prove that the selection-free steady state genetic algorithm analyzed by Sutton and Witt (GECCO 2019) takes an expected number of Ω(2n/n) iterations to find any particular target search point. This bound is valid for all population sizes μ. Our result improves over the previous lower bound of Ω(exp(nδ/2)) valid for population sizes μ=O(n1/2δ), 0<δ<1/2.

Introduction

The theory of evolutionary algorithms (EAs) has produced a decent number of mathematically proven runtime analyses. They explain the working principles of EAs, advise how to use these algorithms and how to choose their parameters, and have even led to the invention of new algorithms. We refer to [1], [3], [9], [10] for introductions to this area.

Due to the complexity of the probability space describing a run of many EAs, the majority of the runtime analyses regard very simple algorithms. In particular, there are only relatively few works discussing algorithms that employ crossover, that is, the generation of offspring from two parents. Among these, again very few present lower bounds on runtimes; we are aware of such results only in [7], [12], [14].

In the most recent of these works, Sutton and Witt [14, Section 3] consider a simple crossover-based algorithm called StSt (μ2) GA0 (made precise in Section 2 below). This steady-state genetic algorithm uses a two-parent two-offspring uniform crossover as only variation operator. The two offspring always replace their parents. There is no fitness-based selection and no mutation in this simple process. Clearly, an algorithm of this kind is not expected to be very useful in practice. The reason to study such algorithms is rather that they allow to analyze in isolation how crossover works (more reasons to study this particular algorithm are described in [14]).

Without fitness-based selection, and thus without regarding the problem to be optimized, one would expect that this algorithm takes an exponential time to find any particular search point of the search space Ω={0,1}n. Surprisingly, this is not so obvious, at least not when working with a particular initialization of the population. Sutton and Witt [14, Theorem 10] initialize the algorithm with μ/2 copies of the string z=(101010) and μ/2 copies of the string z=(010101). They argue that this is a population with extremely high diversity, which could thus be beneficial for a crossover-based algorithm. Sutton and Witt show that their algorithm with this initialization and with population size μ=O(n1/2δ), δ<1/2 a constant, takes an expected number of Ω(exp(nδ/2)) iterations to generate the target string x=(111). Apparently, this lower bound is subexponential for all population sizes. It becomes weaker with increasing population size and is trivial for μ=Ω(n).

By exploiting symmetries in the stochastic process, we improve the lower bound to Ω(2n/n) for all values of μ.

Theorem 1

Let t,μ,nN with μ and n even. Consider a run of the StSt (μ2) GA0 initialized with μ/2 copies of z=(101010) and μ/2 copies of z=(010101). Then the probability that the target string x=(111) is generated in the first t iterations is at most 2t/(nn/2). In particular, the expected time to generate x is at least 14(nn/2)=Ω(2n/n).

Our proof is based on a simple group action or symmetry argument. We observe that the automorphisms of the hypercube {0,1}n (viewed as graph) commute with the operations of the StSt (μ2) GA0. Consequently, if an automorphism σ stabilizes the initial individuals z and z (that is, σ(z)=z and σ(z)=z), then for any x{0,1}n at all times t the probability that the algorithm generates x equals the probability that it generates σ(x).

From this symmetry, we conclude that if B is the set of all x such that there is an automorphism of the hypercube that stabilizes the initial individuals and such that x=σ(x), then at all times the probability that x is generated, is at most 1/|B|. We compute that B has exactly (nn/2) elements. Hence each search point generated by the StSt (μ2) GA0 is equal to x only with probability (nn/2)1. A simple union bound over the 2t search points generated up to iteration t gives the result.

Section snippets

Precise problem statement

The algorithm regarded in [14, Section 3], called StSt (μ2) GA0, is a selection-free variant of a steady state genetic algorithm proposed earlier in [15]. It works with the search space Ω={0,1}n of bit strings of length n, which is a standard representation used in evolutionary computation. The algorithm uses a population of size μ2. Each iteration consists of (i) choosing two different individuals randomly from the population, (ii) applying a two-offspring uniform crossover, and (iii)

Proof of the main result

We now prove our main result following the outline given towards the end of Section 1. We do not assume any prior knowledge on groups and their action on sets.

We view the hypercube {0,1}n as a graph in the canonical way, that is, two bit strings x,y{0,1}n are neighbors if and only if they differ in exactly one position, that is, if H(x,y)=1. A permutation σ of {0,1}n is called graph automorphism if it preserves the neighbor relation, that is, if x and y are neighbors if and only if σ(x) and σ(y

Conclusion and open problems

We proposed an alternative approach to the problem how long the StSt (μ2) GA0 with a particular initialization takes to generate a particular search point [14, Section 3.1]. Our lower bound of order Ω(2n/n), valid for all population sizes μ, is significantly stronger than the previous result, which is at most Ω(exp(n1/4)) and decreases with increasing population size until it is trivial for μ=Ω(n). Our main argument based on group actions is elementary and natural, which gives us the hope that

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This work was supported by a public grant as part of the Investissement d'avenir project, reference ANR-11-LABX-0056-LMH, LabEx LMH.

References (15)

  • Pietro S. Oliveto et al.

    Improved time complexity analysis of the simple genetic algorithm

    Theor. Comput. Sci.

    (2015)
  • Benjamin Doerr et al.

    The impact of random initialization on the runtime of randomized search heuristics

    Algorithmica

    (2016)
  • Benjamin Doerr

    Exponential upper bounds for the runtime of randomized search heuristics

  • Benjamin Doerr

    Probabilistic tools for the analysis of randomized optimization heuristics

  • Axel de Perthuis de Laillevault et al.

    Money for nothing: speeding up evolutionary algorithms through better initialization

There are more references available in the full text version of this article.

Cited by (3)

  • Runtime analysis via symmetry arguments: (hot-off-the-press track at GECCO 2021)

    2021, GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion
View full text