Why does my Keras Conv Net only return 1?

Question

   model = Sequential()
   model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(img_h_w,img_h_w,1)))
   model.add(Conv2D(32, (3, 3), activation='relu'))
   model.add(MaxPooling2D(pool_size=(2,2)))
   model.add(Dropout(0.25))
   model.add(Flatten())
   model.add(Dense(128, activation='relu'))
   model.add(Dropout(0.5))
   model.add(Dense(1, activation='softmax'))
   model.compile(loss='binary_crossentropy', optimizer=keras.optimizers.SGD(lr=0.01),metrics=['accuracy'])
   model.fit(x_train, y_train, batch_size=100, epochs=3, verbose=1)
   score = model.evaluate(x_test, y_test, verbose=0)

Running this returns

2500/2500 [==============================] - 5s 2ms/step - loss: 7.5694 - acc: 0.5252
Epoch 2/3
2500/2500 [==============================] - 5s 2ms/step - loss: 7.5694 - acc: 0.5252
Epoch 3/3
2500/2500 [==============================] - 6s 2ms/step - loss: 7.5694 - acc: 0.5252

Which shows the accuracy is at 50%, and when I look at the predictions made, I see it only outputs 1. It is meant to classify images into 0 or 1.

Answer 1

It is only outputting 1 because softmax makes no sense when given a single input. Softmax computed with only one input $x$ is equivalent to $\frac{e^x}{e^x} = 1$.

In other words, the line

model.add(Dense(1, activation='softmax'))

is wrong. Use sigmoid instead:

model.add(Dense(1, activation='sigmoid'))

or read more about one-hot encoding your expected output and the helper function https://keras.io/utils/#to_categorical.

Answer 2

For binary classification you can use softmax as the output layer but you should consider that if you do so, your last layer have to have two neurons, each corresponds to one specific class. Moreover, you have to use categorical_crossentropy as the loss function. Change the following code:

model.add(Dense(1, activation='softmax'))
model.compile(loss='binary_crossentropy', optimizer=keras.optimizers.SGD(lr=0.01),metrics=['accuracy'])

to:

model.add(Dense(2, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer=keras.optimizers.SGD(lr=0.01),metrics=['accuracy'])