Simple Evolving Connectionis Systems (SECoS)

The next example of a theory which we want to investigate is the paradigm of the Simple Evolving Connectionis Systems (SECoS) introduced by Watts and Kasabov in several papers (the first time by Watts (1999)[344], and then later in several papers, e.g. Watts and Kasabov (2000)[343], Watts and Kasabov (2002)[342], Watts and Kasabov (2009)[338]^8.2

Figure 8.10: SECoS Overview

The general idea of a SECoS can be understood as follows (cf. figure 8.10). One part of the SECoS works in an unsupervised manner by taking input values and generating representations of different spatial regions -'clusters', 'categories', 'fields'...- according to their spatial distribution. This is guided by an apriori fixed sensivity threshold $S_{thr}$, which gives a measure 'how big' a category can grow. The other part of SECoS works in a supervised manner by determining the relations between the categories and the output layer according to some desired output vector. In this context functions an output neuron as a kind of a label $o \in LAB$ which associates a certain subset of categories with a single output neuron.

The input vector is feed into the input layer $I$. The values $I_{i}$ of the input vector are the coordinates of this vector and function as the activation values $A_{i}$ of the neurons of the input layer. If one enters a new input vector $I$ into the system then either there are already some neurons in the hidden (= evolving) layer or not. If not, then the first vector will be taken as a starting evolving neuron whose inputs are the n-many input neurons. The weights $W_{i.j}$ with $j$ as the number of the new evolving neuron are attached to each connection and are taken from the activation values $A_{i}$ of the input neurons. Because the activation $A_{j}$ of the neuron $j$ in the evolving layer will be computed by the formula

$$A_{n} = 1 - D_{I.j}$$ (8.7)

$$A_{n} < S_{thr}$$ (8.8)

with $D_{I.j}$ as the distance between the input layer activation values $A_{I}$ and the weight vector $W_{I.j}$ will the activation of an evolving neuron $j$ decrease when the distance of the neuron to the input layer increases. Every time a new evolving neuron is added the new evolving neuron is furthermore connected with its outputs to the given set of output neurons $O$. The outgoing weights $W_{j.O}$ are set to the values of the desired output vector $O_{d}$.

Another cause for the generation of a new evolving neuron is the fact that the distance between $O_{d}$ and $O_{c}$ is above some threshold $E_{thr}$ or that the intended output neuron has not the highest value:

$$\mid\mid O_{d} - O_{c}\mid\mid > E_{thr}$$ (8.9)

with $O_{c}$ as the calculated output vector and $\mid\mid O_{d} - O_{c}\mid\mid$ as the euclidean distance.

If no new evolving neuron have to be generated then the existing weights will be updated. There are two cases: (i) Update of the incoming weights of an evolving neuron and (ii) update of the outgoing weights.