# Probabilities in KERAS

I'm struggle with a problem OF class probabilities (binary, 0 and 1). Dont know why but after 100 epoches the probabilities became 0 or 1 (like the class to predict). Maybe something is not working with the code or I missing something?

Train data is 20.000 rows (more or less), test data is 2000 rows. Each row is a match where 0 is lost and 1 is won.

y_train_binary = keras.utils.to_categorical(Y, 2)
y_test_binary = keras.utils.to_categorical(t_Y, 2)
model = Sequential()
model.add(Dense(40, input_dim=45, activation='relu'))
model.add(Dropout(0.05))
model.add(BatchNormalization())
model.add(Dense(30, activation='relu'))
model.add(Dropout(0.05))
model.add(BatchNormalization())
model.add(Dense(20, activation='relu'))
model.add(Dropout(0.05))
model.add(BatchNormalization())
model.add(Dense(10, activation='relu'))
model.add(Dropout(0.05))
model.add(BatchNormalization())
model.add(Dense(2, activation='softmax'))
keras.optimizers.Adam(lr=0.5, beta_1=0.9, beta_2=0.999, epsilon=0.3)
model.compile(loss='categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])
model.fit(X, y_train_binary, epochs=300, validation_split=0.2, batch_size=10, verbose=0)
prediction_classes = model.predict_proba(t_X)
numpy.savetxt("C:/Users/Megaport/Desktop/foo.csv", prediction_classes, delimiter=",")


I've tried so many parameters (learning rate, epochs, batch size, epsilon, add layer, less layer, different value of dropout) but the problem is the same: probs are not working.

the probabilities of the class are 1 or 0 and not a value between 1 and 0. This happen after 100 epoches.

data test with prediction after 100 epoches is like this:

**RESULT**   VALUE_A     VALUE_B     VALUE_C    **PRED_0    PRED_1**
0           4          5          3           1          0
0           7          4          5           0          1
1           6          7          6           0          1
1           2          3          4           0          1


What I'm looking for:

**RESULT**   VALUE_A     VALUE_B     VALUE_C    **PRED_0    PRED_1**
0           4          5          3           0.65          0.35
0           7          4          5           0.25          0.75
1           6          7          6           0.20          0.80
1           2          3          4           0.30          0.70


Plus, someone could please give me some advice looking at accuracy and loss graph?

## 1 Answer

Try predicting log-probabilities that would give you bigger seperation (i.e. values between -inf and 0) I Would clip these values (so that you dont take log from 0 which would be -inf but something like if 0 than log(0.0001) For example:

y_pred = estimator.predict_proba(X.drop(columns=self.drop_columns))
log_prob = lambda x: np.log(np.clip(x, 0.001, 0.999)).reshape(-1, 1)
y_pred = np.hstack([log_prob(y_pred[:, 0]), log_prob(y_pred[:, 1])])