Skip to content

Particle Swarm Optimization

Introduction

Particle swarm optimization works by initializing a number of positions in the search space, which move according to their own inertia, the attraction to their own known best positions and the global best position.

Example

from gradient_free_optimizers import ParticleSwarmOptimizer

...

opt = ParticleSwarmOptimizer(search_space, population=5)
opt.search(objective_function, n_iter=100)

About the implementation

The particle swarm optimizer initializes multiple particle classes, which inherit from the hill-climbing class. In the current version all movement-calculations are contained in the particle class. Particles have a velocity, which they maintain because of their inertia even without attraction to other particles via cognitive_weight or social_weight.

The particles use the following parameters and vectors to move through the search space:

  • \(\omega\) => inertia
  • \(c_k\) => cognitive_weight
  • \(c_s\) => social_weight
  • \(r_1\) \(r_2\) => random floats between 0 ... 1
  • \(p_n\) => current position of the particle
  • \(p_{best}\) => best position of the particle
  • \(g_{best}\) => best position of all particles

The velocity of a particle is calculated by the following equation:

\[ v_{n+1} = \omega \cdot v_n + c_k \cdot r_1 \cdot (p_{best}-p_n) + c_s \cdot r_2 \cdot (g_{best} - p_n) \]

Since the particle is always moving for the same single timeframe we can just add the velocity and the current position of the particle to get the new position.

Parameters

population

Size of the population for population-based optimization algorithms. A member of the population is called single optimizer, individual or particle depending on the type of algorithm. Each member of the population is a separate optimizer class with information about the positions and scores of the optimizer and all methods to perform the iteration and evaluation steps.

All population based optimizers in this package calculate the new positions one member at a time. So if the optimizer performs 10 iterations and has a population size of 10, then each member of the population would move once to a new position.

  • type: int
  • default: 10
  • typical range: 4 ... 25

inertia

The inertia of the movement of the individual optimizers in the population.

  • type: float
  • default: 0.5
  • typical range: 0.25 ... 0.75

cognitive_weight

A factor of the movement towards the personal best position of the individual optimizers in the population.

  • type: float
  • default: 0.5
  • typical range: 0.25 ... 0.75

social_weight

A factor of the movement towards the global best position of the individual optimizers in the population.

  • type: float
  • default: 0.5
  • typical range: 0.25 ... 0.75

rand_rest_p

Probability for the optimization algorithm to jump to a random position in an iteration step. It is set to 0 per default. The idea of this parameter is to give the possibility to inject randomness into algorithms that don't normally support it.

  • type: float
  • default: 0
  • typical range: 0.01 ... 0.1