Creating a neural network, composed of n times a different network. Is it possible?

Question

I'm currently working on a project with a bunch of data of devices that can either belong to people, or not. The ultimate goal is to estimate a number of people detected. Sadly, it is impossible to get labels for which device should be classified as human and which shouldn't. So a basic binary classification network will not be trainable.

I do however have for each moment in time a number of people, that can be correlated to the number of human-devices that should be detected (it is not the exact sum of devices that should be detected because we won't be able to detect all).

So... I though of this weird construct, where I have a binary classification neural network (green box in the picture) copied n times along itself, and then have its outputs propagate to an outer network which will perform a regression using both the outputs of the classifiers and some other variables (pink) that will determine the scaling (for example, if only 50% of the humans are detectable using the devices, then the scaling will have to be 2. But this is a non-linear function so it'll also be determined by the NN).

The green neural networks should all share the same weights, because they will perform identical work.

Another possibility would be to use the pink scaling variables from the outer (red) NN, to the green-NN's, and then let it result in a non-binary value, but more like a scaled value. So in the same example as above, a 1 would return 2.

Would this be possible in theory? Has it been done before? What would be the best approach for this?

I already had a talk with my adviser about this, but he was very skeptical about it.

Answer 1

I think it's possible. I don't know how well it will work out for you, but I think it's technically possible (which is the best kind of possible).

For starters, I would suggest using the functional API for Keras for this kind of thing. It makes it super easy to do composition of networks like this, and residual networks like this implementation on GitHub are composed in a similar fashion. Something like the following might get you going (note this isn't meant to be a full solution, sorry):

from keras.layers import Input, Dense
from keras.models import Model

def single_net(input_shape, filters):
    def f(input):
        h0 = Dense(filters, activation='relu')(input)
        h1 = Dense(filters, activation='relu')(h0)
        prediction = Dense(2, activation='softmax')(h1)
        return prediction
    return f

input_a = Input(shape=input_shape)  # whatever your input shape is
input_b = Input(shape=input_shape)
# ... repeat for however many people you have ...
shared_net = single_net(input_shape, filters)  # define once
network0 = shared_net(input_a)
network1 = shared_net(input_b)
# ... again repeat for each person
combined_nets = keras.layers.concatenate([network0, network1, ...])
dense = Dense(output_shape, activation='relu')(combined_nets)  # or however you want to combine them
model = Model(inputs=[input_a, input_b, ...], outputs=dense)
model.compile(optimizer='adam', loss='binary_crossentropy')  # whatever optimizer, loss you like
model.fit([data_a, data_b, ...], labels, ...)

By defining your repeated network only once (shared_net above) and then using that same object for each input, you are sharing those layers across all of those inputs. Basically when you back-propagate, all inputs/outputs will train a single instance of the network, even though you're basically using duplicates of it for each input.

The big question is how do you combine all of your individual models, and how do you deal with changing numbers of models. I don't know how to deal with an uncertain number of models, but I suggest looking at the shared layers section of the Keras functional API documentation for some help on combining, and dig in to some examples on GitHub. What's shown here ought to get you started, but there are probably some errors hiding throughout. Like I said, not a complete answer, sorry.