# Creating a neural network, composed of n times a different network. Is it possible?

I'm currently working on a project with a bunch of data of devices that can either belong to people, or not. The ultimate goal is to estimate a number of people detected. Sadly, it is impossible to get labels for which device should be classified as human and which shouldn't. So a basic binary classification network will not be trainable.

I do however have for each moment in time a number of people, that can be correlated to the number of human-devices that should be detected (it is not the exact sum of devices that should be detected because we won't be able to detect all).

So... I though of this weird construct, where I have a binary classification neural network (green box in the picture) copied n times along itself, and then have its outputs propagate to an outer network which will perform a regression using both the outputs of the classifiers and some other variables (pink) that will determine the scaling (for example, if only 50% of the humans are detectable using the devices, then the scaling will have to be 2. But this is a non-linear function so it'll also be determined by the NN).

The green neural networks should all share the same weights, because they will perform identical work.

Another possibility would be to use the pink scaling variables from the outer (red) NN, to the green-NN's, and then let it result in a non-binary value, but more like a scaled value. So in the same example as above, a 1 would return 2.

Would this be possible in theory? Has it been done before? What would be the best approach for this?

• For variable-length data I suggest you to use RNNs or CNNs. Usually end-to-end networks perform better than hand-constructed pieces merged together. Also the loss function won't be clear in this example. How would you do backpropagation on this? – Bloc97 Dec 21 '18 at 16:29
• @Bloc97: The data isn't variable length per se. I think it should be possible to extract the features in a way so that they always contain formation about a fixed number of devices. However, there is no correlation between the regression and a single device. There is however, a correlation between a bunch of devices (let's say the devices of the last minute?) and the output number. That's why I thought this method was necesarry. I've looked into RNN's and LSTM's (because it is time-data I'm working with indeed), but I'm not sure in what way I would implement it all in this case. – Opifex Dec 21 '18 at 16:37
• An end-to-end RNN should achieve the exact same result as this, and would be easier to construct, train and diagnose. You can think about it mathematically. The unrolled RNN shares weights, and is able to keep track of features that it deems necessary to count the number of humans at each step, then outputs the ratio of human vs non-human devices. It should be easy for an LSTM/GRU network to learn this function. – Bloc97 Dec 21 '18 at 16:43
• But it is just a suggestion to save you some time, you are free and encouraged to try your own method(s). We will never know if it works until someone tries it or proves it formally. – Bloc97 Dec 21 '18 at 16:50

I think it's possible. I don't know how well it will work out for you, but I think it's technically possible (which is the best kind of possible).

For starters, I would suggest using the functional API for Keras for this kind of thing. It makes it super easy to do composition of networks like this, and residual networks like this implementation on GitHub are composed in a similar fashion. Something like the following might get you going (note this isn't meant to be a full solution, sorry):

from keras.layers import Input, Dense
from keras.models import Model

def single_net(input_shape, filters):
def f(input):
h0 = Dense(filters, activation='relu')(input)
h1 = Dense(filters, activation='relu')(h0)
prediction = Dense(2, activation='softmax')(h1)
return prediction
return f

input_a = Input(shape=input_shape)  # whatever your input shape is
input_b = Input(shape=input_shape)
# ... repeat for however many people you have ...
shared_net = single_net(input_shape, filters)  # define once
network0 = shared_net(input_a)
network1 = shared_net(input_b)
# ... again repeat for each person
combined_nets = keras.layers.concatenate([network0, network1, ...])
dense = Dense(output_shape, activation='relu')(combined_nets)  # or however you want to combine them
model = Model(inputs=[input_a, input_b, ...], outputs=dense)
model.compile(optimizer='adam', loss='binary_crossentropy')  # whatever optimizer, loss you like
model.fit([data_a, data_b, ...], labels, ...)


By defining your repeated network only once (shared_net above) and then using that same object for each input, you are sharing those layers across all of those inputs. Basically when you back-propagate, all inputs/outputs will train a single instance of the network, even though you're basically using duplicates of it for each input.

The big question is how do you combine all of your individual models, and how do you deal with changing numbers of models. I don't know how to deal with an uncertain number of models, but I suggest looking at the shared layers section of the Keras functional API documentation for some help on combining, and dig in to some examples on GitHub. What's shown here ought to get you started, but there are probably some errors hiding throughout. Like I said, not a complete answer, sorry.

• This looks very interesting indeed! I'm not sure if the number of models need to be variable in my case. I might be able to implement is so that there's always the same number of device-inputs. Thanks! – Opifex Dec 21 '18 at 16:14
• Glad to hear it! Feel free to upvote, and if it ends up working out for you, mark it as accepted. – Engineero Dec 21 '18 at 16:29
• I tried, but it doesn't let me because I need to have 15 reputation :) But I will upvote it once I'm able to, because it was a very helpful solution indeed! – Opifex Dec 21 '18 at 16:31