# Why does my neural net always converge on a trivial solution?

As part of a more elaborate project, I am trying to get a rather simple network to distinguish between 5 very distinct classes of 1D signals (the members of each class is just the same bit of signal copied over 1000 times with a slight bit of noise added). And failing.

Below is the network setup, and the output. As far as I can tell (through matplotlib), I haven't done anything really silly with data (such as flip the dimensions, for instance). Does anyone have a suggestion for what could be going wrong?

I apologize for not generating dummy data within the script, I'm still much more comfortable in matlab, so it was easier to just port some dummy data from there.

model = Sequential()

model.summary()

data=earEEG['data'][:,:]
data=np.transpose(data)
data=data[:,:,None]

labels=earEEG['labels']
labels= keras.utils.to_categorical(labels)

sgd=keras.optimizers.SGD()
model.compile(loss='categorical_crossentropy',
optimizer=sgd,
metrics=['accuracy'])

hist=model.fit(data, labels, epochs=10, batch_size=50,class_weight='auto',shuffle='True')


And output:

Layer (type)                 Output Shape              Param #
=================================================================
S2 (Flatten)                 (None, 6000)              0
_________________________________________________________________
F1 (Dense)                   (None, 40)                240040
_________________________________________________________________
F2 (Dense)                   (None, 10)                410
_________________________________________________________________
output (Dense)               (None, 5)                 55
_________________________________________________________________
softmax (Activation)         (None, 5)                 0
=================================================================
Total params: 240,505.0
Trainable params: 240,505.0
Non-trainable params: 0.0
_________________________________________________________________
Epoch 1/10
5000/5000 [==============================] - 0s - loss: 12.8945 - acc: 0.2000
Epoch 2/10
5000/5000 [==============================] - 0s - loss: 12.8945 - acc: 0.2000
Epoch 3/10
5000/5000 [==============================] - 0s - loss: 12.8945 - acc: 0.2000
Epoch 4/10
5000/5000 [==============================] - 0s - loss: 12.8945 - acc: 0.2000
Epoch 5/10
5000/5000 [==============================] - 0s - loss: 12.8945 - acc: 0.2000
Epoch 6/10
5000/5000 [==============================] - 0s - loss: 12.8945 - acc: 0.2000


Clearly, the algorithm is just giving the same label to everything (this is also apparent when outputting the actual labels). The input is a 5000 x 6000 matrix, and 5000 labels. As the algorithm does not complain about dimensional mismatch, I feel pretty confident that the data is not being transposed or similar. As an easy test, I tried feeding the same dataset to a pca+decision tree in matlab, and got a perfect score.

Classes are perfectly balanced, by the way.