# Large amount of Sigmoid outputs are ones and zeros

I have Keras neural network for binary classification with final layer having one output with Sigmoid activation. I have noticed that large amount of output numbers are strictly one or zero (rather than between 0 and 1 as expected). What could be the reason for this?

At first I thought maybe network is 100% sure about accuracy of these numbers, but noticed that some of these predictions are actually incorrect.

Edit:

Model:

model = tf.keras.models.Sequential()

model.add(tf.keras.layers.Dense(32, activation = tf.nn.relu, input_shape=(X_train.shape[1],)))

model.add(tf.keras.layers.Dense(64, activation = tf.nn.relu))

model.add(tf.keras.layers.Dense(32, activation = tf.nn.relu))

model.add(tf.keras.layers.Dense(1, activation = tf.nn.sigmoid))

model.compile(optimizer = tf.keras.optimizers.Adam(), loss = 'binary_crossentropy', metrics = ['accuracy', roc_auc])

model.fit(X_train, y_train, epochs=10, validation_data=(X_test, y_test))

y_pred = model.predict(X_test)

yhat = []

for pred in y_pred:
if pred >= 0.7:
yhat.append(1)
else:
yhat.append(0)


Data consists of 8 columns (features) and 270 000 rows (9th column is 'y' column). Out of these 270,000 rows only 9% contained labels of class 1 (rest were zero), so I downsampled the data (just removed bunch of data with label 0), trained the model and then did the prediction on full data. I modified how ones and zeros were determined by Sigmoid though, I changed threshold from 0.5 to 0.7 (which was the ROC score I got on downsampled data)

• Can you share some more detail on your network structure and the data? – serali Nov 15 '19 at 10:37
• @serali I have edited the question – Ach113 Nov 15 '19 at 10:44
• Hi, please avoid posting images of code / outputs which can be pasted as text in the question. – Romain Reboulleau Nov 15 '19 at 10:58
• @Ach113 Could you share the actual code you use to build the network and the way you check the values of the final sigmoid? Its probably a small mistake in the code, it is impossible to tell by looking just at the summary. – serali Nov 15 '19 at 11:27
• @serali Sure, updated the post – Ach113 Nov 15 '19 at 11:31