Skip to content

Differential Evolution

Introduction

Differential evolution is an optimization technique that iteratively improves a population of candidate solutions by combining and perturbing them based on their differences. It generates new candidates by adding the weighted difference between two population members to a third member, creating trial solutions that are evaluated for their fitness. If a trial solution is better than the target it replaces, ensuring continual improvement. This method is effective for optimizing complex, nonlinear, and multimodal functions where traditional gradient-based methods are impractical.

Example

from gradient_free_optimizers import DifferentialEvolutionOptimizer

...

opt = DifferentialEvolutionOptimizer(search_space, mutation_rate=0.8)
opt.search(objective_function, n_iter=100)

About the implementation

The differential evolution optimizer inherits from the evolutionary algorithm class. Therefore it is similar to the genetic algorithm or evolution strategy, like using a crossover step to mix individuals in the population by e.g. discrete recombination. But it sets itself apart in the iteration loop by always doing a mutation step and crossover step to mix the target- and mutant-vector.

Parameters

population

Size of the population for population-based optimization algorithms. A member of the population is called single optimizer, individual or particle depending on the type of algorithm. Each member of the population is a separate optimizer class with information about the positions and scores of the optimizer and all methods to perform the iteration and evaluation steps.

All population based optimizers in this package calculate the new positions one member at a time. So if the optimizer performs 10 iterations and has a population size of 10, then each member of the population would move once to a new position.

  • type: int
  • default: 10
  • typical range: 4 ... 25

mutation_rate

Mutation Rate (F), also known as the differential weight, is a parameter in Differential Evolution (DE) that controls the amplification of the differential variation between individuals. It is a scaling factor applied to the difference between two randomly selected population vectors before adding the result to a third vector to create a mutant vector. The mutation rate influences the algorithm's ability to explore the search space; a higher value of F increases the diversity of the mutant vectors, leading to broader exploration, while a lower value encourages convergence by making smaller adjustments. The typical range for F is between 0.4 and 1.0, though values outside this range can be used depending on the problem characteristics.

\[ \mathbf{v}_{i,G+1} = \mathbf{x}_{r1,G} + F \cdot (\mathbf{x}_{r2,G} - \mathbf{x}_{r3,G}) \]
  • type: float
  • default: 0.9
  • typical range: 0.4 ... 1.0

crossover_rate

Crossover Rate (CR) is a parameter in the Differential Evolution (DE) algorithm that controls the probability of mixing components from the target vector and the mutant vector to form a trial vector. It determines how much of the trial vector inherits its components from the mutant vector versus the target vector. A high crossover rate means that more components will come from the mutant vector, promoting exploration of new solutions. Conversely, a low crossover rate results in more components being taken from the target vector, which can help maintain existing solutions and refine them. The typical range for CR is between 0.0 and 1.0, and its optimal value often depends on the specific problem being solved.

\[ u_{i,j,G+1} = \begin{cases} v_{i,j,G+1} & \text{if } \text{rand}_j(0,1) \leq CR \text{ or } j = j_{\text{rand}} \\ x_{i,j,G} & \text{otherwise} \end{cases} \]
  • type: float
  • default: 0.9
  • typical range: 0.0 ... 1.0