0
$\begingroup$

my input is a 200 dims vector, which is generated by mean of the word2vector of all words of a article, my output is a 50 dims vector,which is generated by the LDA results of a article I want to use mse as the loss function,but the value of the loss always be 0 my code as follows:

model = Sequential()
model.add(Dense(cols*footsize, 400,init = "glorot_uniform"))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Dense(400, 400,init = "glorot_uniform"))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Dense(400, 50,init = "glorot_uniform"))
model.add(Activation('softmax'))
model.compile(loss='mse', optimizer='rmsprop')

And regardless of the number of epochs, the loss is always 0

$\endgroup$
1
$\begingroup$

First, is your output a one-hot vector of predicted classes? i.e: class one is [1, 0, 0, ...] and class two is [0, 1, 0, 0, ...].

If so, then using softmax activation at the output layer is acceptable and you are doing a classification problem. If you are doing a classification problem (one-hot output) you cannot use MSE as the loss, use categorical cross-entropy.

Softmax scales the output so that the number given is a predicted probability of a certain class. Wikipedia here: https://en.wikipedia.org/wiki/Softmax_function

If you are expecting the output vector to be real numbers then you need to use linear activation on your output neurons.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.