Forward propagation in complicated neural networks

Question

In a typical neural network you have a certain number of neurons in each layer, all neurons have incoming connections from each neuron in the previous layer (and visa-versa). This I have no problem figuring out how to propagate.

However when you have a network that doesn't have the neurons in layers (I'm looking into the NEAT algorithm); where neurons are just connected based on genetic algorithm (random creation - then testing of fitness) there isn't a simple way of figuring how to forward propagate, for example;

1 input neuron - connected to neurons 2 and 3
Neuron 2 is also connected to neuron 3
Neuron 3 is connected to the output.

Now in this case, if I just go through numerically then everything works out just fine. However with this subtle change;

1 input neuron - connected to neurons 2 and 3
Neuron **3** is also connected to neuron **2**
Neuron **2** is connected to the output.

The going through; 1, 2, then 3 would mean neuron 2 doesn't include the input from neuron 3. I can see that there's a brute force way through this, but is there a better, more efficient way?

Answer 1

You can do topological sorting. For example, here is some pythonic pseudocode

def order(current_neuron, level):
    current_neuron.level = max(level, current_neuron.level)
    for child in sorted(current_neuron.children, key=children level, reverse=True):
        order(child, level + 1)

for input_neuron in input_neurons:
    order(input_neuron, level=0)

(Take this pseudo code with a grain of salt. This is just what came directly to my mind; it is not tested.)

Then you can calculate the activations in order of node.level.