Genetic Programming (GP) is similar to a specialization of GAs with the representation of individuals as computer programs or genetic programs. But, through a very active research community, GP has become very dif- ferentiated from GAs. The precise early roots of GP are hard to distin- guish. In 1954, Nils Aall Barricelli used programs as the representation for evolutionary algorithms and during the 1960s, 1970s and 1980s the core ideas and mathematics behind GP were developed by various researchers including: Turing [118], Price [95] and Holland [34]. Researchers to ap- ply early versions of tree-based GP include: Forsyth [25], Cramer [17], De Jong [35] and Bickel and Bickel [9]. Despite this early work, Koza [40, 41] originated many of the standard constructs still found in today’s canonical GP.
The broader method of GP has the key feature of complex representa- tions for individuals. Historically these representations described simpli- fied runnable computer programs but the term “numerical expression” is often more apt than “program” since the programs often involve few of the typical constructs found in computer programs; typically these genetic programs use no loops or subroutines. However voluminous research has focussed on the best way to add such constructs [69, 14, 54].
2.4. OVERVIEW OF GENETIC PROGRAMMING 19
2.4.1
Representation of individuals
GP features an ever growing set of representations for individuals. While GAs typically use bit-strings, GP researchers are perhaps freer to invent more complex representations. Despite the name, some modern represen- tations of genetic programs bear no resemblance to computer programs of any sort. Some commonly used representations for genetic programs include:
• Tree [41] - the structure for the original genetic programs was as Lisp S-expressions or trees of functions and terminals, used in tree-based GP, and this form is still most popular.
• Grammar tree [123] - distinct from tree-based GP is grammar-based GP. Each individual is a partial derivation tree, which may be used to derive a tree-based GP genetic program.
• Linear [11] - each individual in Linear GP (LGP) is a sequence of in- structions, similar to those of assembly language. LGP uses a register set as both input and output.
• Graph [55, 21, 66] - each individual is a graph of functions and ter- minals, more generally than the trees of tree-based GP. LGP genetic programs may also be interpreted as graphs.
• Final representation - recently, GP researchers have been foregoing the step from genetic program to final output and have been evolv- ing genetic programs of the final representation directly. An example is the evolution of electronic circuits [43].
This thesis focuses on tree-based GP only. The sections after this one and the remaining thesis, exclusively reference tree-based GP.
2.4.2
Tree-based GP individuals
A genetic program in tree-based GP is a tree of functions and terminals. Functions are the internal nodes of the tree and terminals are the leaf nodes of the tree. The tree structure of a genetic program means that any number of inputs may be arbitrarily processed by component functions and one output produced. The GP system may evaluate the tree in the same way as a Lisp program is evaluated.
The functions and terminals of a program tree may be evaluated to produce an output. Terminals have no children and their output values depend on stored value, as in the case of numeric terminals, or informa- tion from the “world”, as in the case of feature terminals. Functions have children and use the outputs of those children to determine their own out- put. An algorithm may evaluate each node by ensuring children evaluate before parents. The evaluated output of the root node is the output of the genetic program as a whole.
2.4.3
Generation of an individual
There are a variety of ways to generate the tree for a genetic program, including:
• Full - randomly assign the nodes of each level, starting at the root. If the desired depth is d, then all nodes at depths less than d are functions and all nodes at d are terminals.
• Grow - similar to full, but nodes at depths less than d may also be assigned as terminals with a given probability.
2.4.4
Generation of the initial population
There are a variety of ways to generate the initial population of genetic programs, including:
2.4. OVERVIEW OF GENETIC PROGRAMMING 21 • Half-and-half [41] - this method generates half of the genetic pro-
grams of the population using full and the other half by grow. • Ramped [41] - this method varies the desired depth of the genetic
programs linearly from some minimum for the first genetic program to some maximum for the last genetic program.
• Ramped half-and-half [41] - a commonly used combination of “ramped” and “half-and-half”
2.4.5
Genetic operators on individuals
Publications have focussed on improving the genetic operators in GP [47, 133] but the original set of operators is still most common:
• Subtree mutation - produces a single child from the single parent with one randomly selected subtree replaced by a randomly gener- ated subtree.
• Subtree crossover - produces two children from the two parents, each sourced from alternate parents. Crossover first selects a location in the tree of each parent and produces two children each with the main body of one parent with its selected subtree replaced by the selected subtree of the other parent.
2.4.6
Current developments in genetic programming
GP is being actively developed in several ways:
• Co-evolution [127, 27] - uses two or more concurrent evolutions, in which each evolution depends on each other for its fitness function. That is, each population learns off the other.
• Multi-objective [73, 24] - in which the fitness function produces a vector of numbers, rather than a single real-valued fitness.
• Parallelization [26, 44] - GP need not simply repeatedly produce one population from the last. Recent research has focussed on paralleliz- ing GP over multiple populations.
• Variation of parameters during evolution [119, 33] - The optimal set- ting for each of the many parameters to GP need not be constant through each evolution or between evolutions. Research has focussed on adjusting parameters during evolution.
• Bloat control and removal of underused code [120, 125] - bloat or the tendency of genetic programs in later generations to be larger, has long been a problem in GP.
• Utilizing GPUs (GPGPU) [50, 68] - a new topic in GP is the use of highly parallel architectures such as GPUs which are most often as- sociated with graphics.
• Analysis of GP - to direct the improvements to GP a large amount of recent research focuses on the analysis of GP:
– Building blocks - hypothesized as a loose entity by Goldberg in 1985 [29], the term building-block has come to mean any one of many concrete structures found in genetic programs. Much recent research uses building-blocks to analyze improvements in performance.
– Research of schemata and building-blocks in GP [39, 117, 45, 36] - research looks to schema theory to explain the power of GP.