Development is the natural process of cell-division, differentiation, apoptosis, growth and morphogenesis [404, 211] that turns a single zygote cell into a mature organism and maintains it in a healthy structural and functional condition. In a sense, development can be seen as the machinery that determines the behaviour of an organism either directly or indirectly through generating some functional structures (such as nervous system). Development is a parallel, decentralised, and self-organised process [103]. Rust et al. count variation, adaptation, regulation, modularity, and robustness as emergent features of biological development [321].
As a more bio-plausible approach to evolutionary computation, artificial development (or compu- tational development) is an abstracted model of biological development believed to be able to introduce emergent properties such as scalability, adaptability, robustness, self-organisation, modularity, growth, regularisation, regeneration, fault-tolerance, and self-repair to the products of evolutionary algorithms [103, 261, 211]. Such systems that use evolutionary algorithms and artificial development are some- times called Evolutionary Developmental Systems or Evo-Devo systems, which come from the literature in biology [298].
Several mathematical and computational models of development have been introduced by re- searchers. Among them, Turing’s Reaction-Diffusion systems [368], Meinhardt’s Activator-Inhibitor models [258, 259], Kauffman’s Random Boolean Networks [189], and Lindenmayer systems (L- systems) [221] have been studied extensively and applied to different domains [211].
Rewriting Systems
Lindenmayer systems (L-Systems) [221], its variations (such as bracketed, stochastic, parametric, and context-sensitive L-Systems), and similar rewriting systems are used for developing 1,2 and 3 dimen- sional arrays [222], which can be then interpreted into 1,2, and 3D shapes and structures, plants, graphs, circuits, hierarchical structures, and modular networks [296, 103, 260, 40].
Context-sensitive parametric bracketed L-Systems can be used to imitate cell signalling, regulation and protein diffusion [103]. One limitation of the rewriting systems is that modelling cell migration needs an external and implausible process (deleting one symbol and inserting it at another location) [103]. Cell migration can be modelled much easier in detailed models. Rewriting systems were originally devised for simulating the biological developmental processes in self-similar organism such as platens and they are not easy to design.
Cellular Systems
The cell is the fundamental building block of life. Each cell is separated from its environment and other cells by a membrane (and an extra cell wall in some cells) that defines its boundary and controls the matter and energy exchange with the environment. Each cell also has a genome that provides the instructions for functions and development of the cell and in turn maybe a whole multicellular organism. The densities of different chemical molecules inside the cell determine its state and the molecules that pass through the cell membrane can be used for signalling and nutrition [404]. Artificial cellular systems have a set of variables presenting the state of each cell [103]. A number of equations, instructions or other rules define the state transition function, describing how the cell state evolves through time. A set of rules also define how the cell communicates with its environment and other cells [103].
Cellular Automata (CA) is the simplest form of artificial cellular systems. It was first introduced by von Neumann in [393] and then studied extensively by Wolfram [402] and others. It is essentially a 1,2, or 3 (or more) dimensional array of cells with a discrete cell space of arbitrary neighbouring (grid, honeycomb, etc.), a finite state set, a discrete time variable, and a state transition function that defines the next states of the cell given the current state of the cell and states of a limited number of its neighbours. There are also variations of CA such as asynchronous, probabilistic, and non-homogenous CAs. In a non-homogenous CA, the state transition function can be time or space dependent. CAs have been used for modelling development among many different processes in nature and technology [103].
It is also possible to use finer resolutions or continuous time and state variables, signalling, and diffusion, or different levels of cellular systems that work together to model biological cellular systems more accurately [103, 211]. Cellular structures can be also used effectively in routing signals both in digital [364, 377] and analog [92, 93] systems. As diffusion and signalling are significant processes in biological development, their bio-plausible simulation is important in this study. Therefore, some diffusion models are reviewed here.
Diffusion Models
Bio-inspired multi-cellular developmental systems usually need to model the diffusion of proteins and other chemicals. These models can be analytical or numerical. In an analytical model, the concentration
of a protein can be calculated as a function of the position of all the protein diffusers (in vicinity of the point in question) and their strength, the diffusion coefficient of the protein in the substrate, and the decay rate of the protein. For example Kumar et. al. model the contribution of each diffuser to the concentration at a certain point (C) with Euclidian distance d from the source (diffuser) using an equation of the form [210]:
C = Cs· e
−d2
2D2 (2.15)
where D is the diffusion coefficient and Csis the concentration at the source, resulting in a Gaussian
radial base function.
However, Rust et al. model diffusion using an equation of the general form [321]:
C =Cs
dω (2.16)
where ω ∈ {1, 2, 3, 4} is the decay factor (typically 1). Other models exist that also take time into consideration [89].
In numerical models the differential equations governing the process of diffusion is approximated using the Euler method by discretisation of time and/or space. For example Miller used the update rule of general form [261]: Ct+1= Ct ω + 1 |N |ω X i∈N Ci t (2.17)
where Ctis the concentration in a cell at time step t and Cti, i ∈ N are concentrations in the neighbouring
cells.
Deterministic or stochastic computation in form of parallel or serial arithmetic implementations can be used to approximate such an update rule [15]. Sometimes very simplistic degenerated forms of this update rule are used to minimise computing resources. For example Roggen used this update rule [311]:
Ct+1= max
i∈N (C
i
t− 1, 0) (2.18)
in a hardware-based implementation to avoid division operations and multiple additions.
These models are different in terms of bio-plausibility, computational cost and providing the concentration gradient information with a clear trade-off between computational complexity and bio- plausibility.
Cellular Developmental Systems
Cellular developmental systems are similar to context sensitive rewriting systems in many ways. Their state transition functions of neighbour states are similar to rewriting rules. Cell states and cell space in CA resembles the symbols alphabet and cells in rewriting systems. However, in CAs and most of the cellular systems the cell space and number of cells are predefined and fixed, while rewriting systems are capable of creating new cells and deleting old ones. Although this can be an advantage of the rewriting systems, it also makes it quite challenging to implement them in a parallel piece of hardware with limited resources. CAs (and to some extent other cellular systems) ,on the other hand, are very easy to implement in parallel hardware and particularly in FPGAs.
Biological development can be modelled at different levels of abstraction. One extreme is to look at very high-level dynamics of organ formation or cell duplication. Rewriting systems such as Lin- denmayer systems (L-systems) [221] are very good examples of such high-level approaches. It is also possible to delve into the details of the biological development process looking for the mechanisms and structures and imitate them to achieve a higher level of bio-plausibility. Multicellular systems of gene regulatory networks (GRNs) with protein folding, diffusion, simulated chemistry, and gene expression and regulation, are examples of the later extreme. There is a spectrum of different models between these two extremes, which have been developed and applied to different applications with a diverse degree of success and performance match [103, 211]. Here we focus on artificial evolutionary developmental (evo-devo) models with an emphasis on bio-plausible models and those that are useful in developing circuits and particularly neural networks.