# Why does my loss value start at approximately -10,000 and my accuracy not improve?

I'm developing a multi-label classifier using the Keras library, but I am stuck with a relatively low accuracy of about 2% and my loss value per epoch is around -10,000 with little change between epoch. I'm relatively new to Keras but need to use it for my university work, and I am wondering what is going wrong.

My model currently looks as follows:

model = Sequential()
model.add(Dense(1024, activation='relu', input_shape=X.shape[1:]))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(3, activation='sigmoid'))

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])

model.fit(X, y, batch_size=20, epochs=100)


In this case, the input_shape[1:] is 4096, a flattened feature array and the multi-label I'm trying to classify has a first value that ranges from 1 - 163, the second value ranges from 1 - 2000, and the third value ranges from 2007 - 2014.

The issue that I am having is while running my model I get a result that does not change during each epoch as is shown below:

I don't know what I'm doing wrong, I'll include how the features and the labels look below in respective order:

[['0.0' ' 0.0' ' 0.0' ..., ' 5.7333' ' 0.0' ' 2.58643']
['0.0' ' 3.12623' ' 0.0' ..., ' 0.0' ' 0.0' ' 2.93147']
['0.0' ' 0.0' ' 0.0' ..., ' 7.49419' ' 0.0' ' 1.55746']
['0.0' ' 0.0' ' 0.0' ..., ' 0.0' ' 0.0' ' 0.3666']
['0.0' ' 4.67996' ' 0.0' ..., ' 0.0' ' 0.0' ' 0.0']]

[['78' '1' '2010']
['78' '1' '2010']
['78' '1' '2014']
['78' '1' '2013']
['78' '1' '2012']]


I'm at a loss of what to try and as I have mentioned, I'm new to Keras and deep learning so any help will be greatly appreciated.

• Also check your loss function and it will be better to one hot your targets and change mode tocategorical and then use categorical loss..Also we need to modify the architecture a bit – Aditya Mar 1 '18 at 17:09

## 1 Answer

You are using the sigmoid activation function on the output layer that squashes the values between 0 and 1, but you are setting the targets outside of that range. I suggest scaling your outputs between 0 and 1 for each output unit. By the way, using a library like conx that is built on Keras would give you a warning about this very issue. For more, see: http://conx.readthedocs.io/en/latest/Getting%20Started%20with%20conx.html#A-Simple-Network