16
$\begingroup$

I have built a LSTM model to predict duplicate questions on the Quora official dataset. The test labels are 0 or 1. 1 indicates the question pair is duplicate. After building the model using model.fit, I test the model using model.predict on the test data. The output is an array of values something like below:

 [ 0.00514298]
 [ 0.15161049]
 [ 0.27588326]
 [ 0.00236167]
 [ 1.80067325]
 [ 0.01048524]
 [ 1.43425131]
 [ 1.99202418]
 [ 0.54853892]
 [ 0.02514757]

I am only showing the first 10 values in the array. I don't understand what do these values mean and what is the predicted label for each question pair?

$\endgroup$
  • 1
    $\begingroup$ I think you have problem in your network .. the probabilities should be in scale 0-1 .. but you have 1.99 !, I think you have something wrong .. $\endgroup$ – Ghanem Nov 16 '18 at 17:01
8
$\begingroup$

The output of a neural network will never, by default, be binary - i.e. zeros or ones. The network works with continuous values (not discrete) in order to optimise the loss more freely in the framework of gradient descent.

Have a look here at a similar question that also shows some code.

Without any kind of tweaking and scaling, the output of your network is likely to fall somewhere in the range of your input, in terms of its nominal value. In your case, that seems to be roughly between 0 and 2.

You could now write a function that turns your values above into 0 or 1, based on some threshold. For example, scale the values to be in the range [0, 1], then if the value is below 0.5, return 0, if above 0.5, return 1.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks, I too thought of using a threshold value to classify the labels. But what should be the basis on which the threshold value decided? $\endgroup$ – Dookoto_Sea Jul 31 '18 at 14:39
  • $\begingroup$ @Dookoto_Sea you have to decide it yourself $\endgroup$ – Jérémy Blain Nov 29 '18 at 9:07
  • $\begingroup$ @Dookoto_Sea Please note that if your label is 0 or 1, your value should be in that range, having a predictions values scales of [0, 2] is intriguing, you need to change your model output $\endgroup$ – Jérémy Blain Nov 29 '18 at 9:09
7
$\begingroup$

If this is a classification problem you should change your network to have 2 output neurons.

You can convert labels to one-hot encoded vectors using

y_train_binary = keras.utils.to_categorical(y_train, num_classes)
y_test_binary = keras.utils.to_categorical(y_test, num_classes)

Then make sure that your output layer has two neurons with a softmax activation function.

model.add(Dense(num_classes, activation='softmax'))

This will result in your model.predict(x_test_reshaped) to be an array of lists. Where the inner list is the probability of an instance belonging to each class. This will add up to 1 and evidently the decided label should be the output neuron with the highest probability.

Keras has this included in their library so you don't need to do this comparison yourself. You can get the class label directly by using model.predict_classes(x_test_reshaped).

| improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ "If this is a classification problem you should change your network to have 2 output neurons. " .. sorry Jah, but he should not, he can do it with one neuron and sigmoid instead of softmax function. $\endgroup$ – Ghanem Nov 12 '18 at 18:32
  • $\begingroup$ @Minion, both methods are essentially equivalent, the thresholding that you would otherwise need to do with a single output neuron is implicitly embedded in the network. Thus providing the binary output. $\endgroup$ – JahKnows Nov 16 '18 at 5:01
  • 1
    $\begingroup$ Yes I kow .. I commented just because he mentioned: "should change your network to have 2 output neurons." .. thanx $\endgroup$ – Ghanem Nov 16 '18 at 16:57
1
$\begingroup$

The predictions are based on what you feed in as training outputs and the activation function.

For example, with 0-1 input and a sigmoid activation function for the output with a binary crossentropy loss, you would get the probability of a 1. Depending on the cost of getting the decision wrong in either direction you can then decide on how you deal with these probabilities (e.g. predict category "1", if the probability is >0.5 or perhaps already when it's >0.1).

From what you describe, you had 0-1 input and presumably assumed a linear activation for your output layer (perhaps with a mean-squared-error loss?). This means, you assumed a regression problem, where the output is directly a predicted number (that can be any real number from in $(-\infty, \infty$). I'd assume that was not what you intended and you might want what I mentioned in the first paragraph instead.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.