0
$\begingroup$

I'm developing a multi-label classifier using the Keras library, but I am stuck with a relatively low accuracy of about 2% and my loss value per epoch is around -10,000 with little change between epoch. I'm relatively new to Keras but need to use it for my university work, and I am wondering what is going wrong.

My model currently looks as follows:

model = Sequential()
model.add(Dense(1024, activation='relu', input_shape=X.shape[1:]))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(3, activation='sigmoid'))

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])

model.fit(X, y, batch_size=20, epochs=100)

In this case, the input_shape[1:] is 4096, a flattened feature array and the multi-label I'm trying to classify has a first value that ranges from 1 - 163, the second value ranges from 1 - 2000, and the third value ranges from 2007 - 2014.

The issue that I am having is while running my model I get a result that does not change during each epoch as is shown below:

enter image description here

I don't know what I'm doing wrong, I'll include how the features and the labels look below in respective order:

[['0.0' ' 0.0' ' 0.0' ..., ' 5.7333' ' 0.0' ' 2.58643']
 ['0.0' ' 3.12623' ' 0.0' ..., ' 0.0' ' 0.0' ' 2.93147']
 ['0.0' ' 0.0' ' 0.0' ..., ' 7.49419' ' 0.0' ' 1.55746']
 ['0.0' ' 0.0' ' 0.0' ..., ' 0.0' ' 0.0' ' 0.3666']
 ['0.0' ' 4.67996' ' 0.0' ..., ' 0.0' ' 0.0' ' 0.0']]

[['78' '1' '2010']
 ['78' '1' '2010']
 ['78' '1' '2014']
 ['78' '1' '2013']
 ['78' '1' '2012']]

I'm at a loss of what to try and as I have mentioned, I'm new to Keras and deep learning so any help will be greatly appreciated.

$\endgroup$
1
  • $\begingroup$ Also check your loss function and it will be better to one hot your targets and change mode tocategorical and then use categorical loss..Also we need to modify the architecture a bit $\endgroup$
    – Aditya
    Mar 1, 2018 at 17:09

1 Answer 1

3
$\begingroup$

You are using the sigmoid activation function on the output layer that squashes the values between 0 and 1, but you are setting the targets outside of that range. I suggest scaling your outputs between 0 and 1 for each output unit. By the way, using a library like conx that is built on Keras would give you a warning about this very issue. For more, see: http://conx.readthedocs.io/en/latest/Getting%20Started%20with%20conx.html#A-Simple-Network

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.