The Derivative Based and Derivative-Free algorithms discussed are all determinis- tic, in that that they have no random components. These algorithms are efficient if the function being optimised has only one optimum. However they show poor performance in finding a global optimum in a complex function which has multi- ple maxima and minima.
The final type of optimisation strategy to be discussed in this thesis is Meta- heuristic algorithms which are a powerful set of algorithms that are often inspired from nature [107]. There are a diverse range of Metaheuristic algorithms such as ant and bee algorithms and particle swarm. This chapter discusses two prominent techniques, Genetic Algorithms and Simulated Annealing. These algorithms are the most well understood and reviewed within the combinational optimisation literature, hence there inclusion in the thesis.
5.3.3.1 Genetic Algorithms
Genetic Algorithms are an optimisation technique pioneered by J.Holland and several collaborators during the 60’s and 70’s. The algorithms are based on the abstraction of Darwin’s evolution of biological systems. There are three main components or genetic operators in a genetic algorithm consisting of, crossover,
mutation and selection of the fittest [107]. Each potential solution to the opti- misation problem is encoded in a string called a chromosome. New solutions are generated by applying the genetic operators, to produce the following results :
1. A crossover of two parent strings will produce offspring i.e. new solutions. This is performed by swapping genes of the chromosomes. Crossover has a typical probability of between 0.8-0.95 of producing a new solution.
2. A mutation is performed by flipping some digits of a string, the typical probability of generating a new solution is somewhere in the range of 0.001- 0.05.
3. The new solutions produced by crossover and mutation are then selected according to their fitness, which results in only the best solutions remaining in the population. Sometimes the best solutions will be passed onto the next generation without much change (elitism).
5.3.3.2 Simulated Annealing
Simulated Annealing (SA) was developed by Kirkpatrick et al. in 1983 [114]. The idea of SA comes from a paper published by Metroplis et al in 1953 [115], which simulated the cooling of a material in a heat bath (commonly known as annealing). If a solid is heated past its melting point and then allowed cool, the structural properties of the solid will depend on the rate of cooling in that :
• A slow enough cool rate will result in large crystals being formed.
• A fast cool rate will result in crystals being formed which contain imper- fections.
The algorithm presented by Metropolis simulated the material as a system of particles and the associated cooling process. The cooling process is simulated by gradually lowering the temperature until the system converges to a steady frozen state. Kirkpatrick et al. took the idea of the Metropolis algorithm and applied it to optimisation problems. The idea being to search for feasible solutions and converge to an optimal solution. The algorithm itself can be explained as :
1. Generate an initial random solution and calculate the value of the objective function.
2. Generate another random solution and calculate the value of the objective function. If the value of the objective function is lower for the new solution than the previous solution (minimisation problem) automatically accept
this new solution, otherwise calculate the probabilityP of accepting a worst solution as: P =exp − c t (5.12)
where c = the change in the objective function and t = the current tem- perature. A random number, Rn between (0,1) is generated. The worse solution will then be accepted or rejected based on whether the probability of accepting a worse solution is greater or less than the generated random number using the following criteria:
• If P > Rn then accept the worse solution.
• If P < Rn then reject the worse solution.
The probability of accepting a worse solution is a function of the temper- ature as well as the change in the value of the objective function. As the temperature is lowered the probability of accepting a worse move is reduced. This is the same as gradually moving to a frozen state in physical annealing. The performance of the Simulated Annealing algorithm is dependent on the cool- ing schedule which consists of four components :
• Starting Temperature - This must be hot enough to allow nearly any random solution to be considered. If this is too low, the end solution will be same or very close to the starting solution.
• Final Temperature - Often the final temperature is set at zero, however this can lead to the algorithm having to run a lot longer than required.
• Temperate Decrement and Iterations at each temperature - After establish- ing a start and end temperature a method needs to be found to lower the temperature. This is normally dependent on the algorithm performing a certain number of iterations at each temperature. The number of iterations should be high enough such that the algorithm reaches equilibrium at each temperature value. Their are many different ways of decrementing the tem- perature. The best results from the authors experience are often obtained using a geometric decrement t=αtwhere α <1
This section of the thesis has given a broad overview of computational optimi- sation and some of the main algorithms in use today. It is apparent from the techniques reviewed in this section, that when using any optimisation technique, the problem to be solved must be clearly identified and a thorough design specifi- cation produced in order to determine the most appropriate optimisation method to use.