My question is about the structure of the network required to solve my problem with fewer data.
I have a sensor device that simply reports the color of the thing it's seeing in front of it. One sensor reports me 4 numbers: Red, Green, Blue, and Alpha. The intensity of the color changes depends on the distance and the thing it's seeing. I have 6 such sensors attached on each side of a small cube. The cube can be moved and rotated using the hand.
I want to predict the position of the cube in space in real-time.
Input: 6 identical sensors, each giving 4 numbers. Total=6*4=24 numbers.
Output: 3 numbers, X, Y, Z, (Position of the cube)
I have my data ready with label XYZ.
Now, train a simple Multi-layer perceptron which takes 24 numbers and output 3 numbers. This is working quite fine but it needs hell lots of data in a cubic meter space to predict accurately.
The problem is about the rotation. I need to rotate and cover 360 degrees for every location for it to predict well.
But I know for the fact that each sensor is identical so I want to share the weight of each sensor. I know that when you rotate the cube 90 degrees, it should not affect the output position at all. So this should mean the order of the sensors is not important. This means I should somehow use
average to merge my sensors' layer. If I use
concatenate it will preserve the order which make the output position changes.
The way I'm doing it is that I feed 4 numbers into a sensor model that are shared among all sensors, get the encoding, add them up, then connect it to Dense layers. Following is the model prototype:
from keras.layers import Input, Dense from keras.models import Model, Sequential sensor1 = Input(shape=(4,)) sensor2 = Input(shape=(4,)) sensor3 = Input(shape=(4,)) sensor4 = Input(shape=(4,)) sensor5 = Input(shape=(4,)) sensor6 = Input(shape=(4,)) sensor_model = Sequential([ Dense(64, activation='relu'), Dense(64, activation='relu'), ]) sensor1_encoding = sensor_model(sensor1) sensor2_encoding = sensor_model(sensor2) sensor3_encoding = sensor_model(sensor3) sensor4_encoding = sensor_model(sensor4) sensor5_encoding = sensor_model(sensor5) sensor6_encoding = sensor_model(sensor6) sensor_encoding = average([ sensor1_encoding, sensor2_encoding, sensor3_encoding, sensor4_encoding, sensor5_encoding, sensor6_encoding, ]) h = sensor_encoding h = Dense(128, activation='relu')(h) h = Dense(128, activation='relu')(h) h = Dense(3, activation='linear') model = Model(inputs=[sensor1, sensor2, sensor3, sensor4, sensor5, sensor6], outputs=[h])
Right now, when I change the
average function to
concatenate, the model loss is lower on both training and validation set which contradicts with my intuition. What is wrong with my thinking?
What do you think? How do I adjust this model in order for it to predict the same position if I rotate 90 degrees while also not suffering from 45 degrees rotation. And also make it in such a way that it does not remove the useful relationship between the inputs.