1
$\begingroup$

I'm struggle with a problem OF class probabilities (binary, 0 and 1). Dont know why but after 100 epoches the probabilities became 0 or 1 (like the class to predict). Maybe something is not working with the code or I missing something?

Train data is 20.000 rows (more or less), test data is 2000 rows. Each row is a match where 0 is lost and 1 is won.

y_train_binary = keras.utils.to_categorical(Y, 2)
y_test_binary = keras.utils.to_categorical(t_Y, 2)
model = Sequential()
model.add(Dense(40, input_dim=45, activation='relu'))
model.add(Dropout(0.05))
model.add(BatchNormalization())
model.add(Dense(30, activation='relu'))
model.add(Dropout(0.05))
model.add(BatchNormalization())
model.add(Dense(20, activation='relu'))
model.add(Dropout(0.05))
model.add(BatchNormalization())
model.add(Dense(10, activation='relu'))
model.add(Dropout(0.05))
model.add(BatchNormalization())
model.add(Dense(2, activation='softmax'))
keras.optimizers.Adam(lr=0.5, beta_1=0.9, beta_2=0.999, epsilon=0.3)
model.compile(loss='categorical_crossentropy', optimizer='Adam', metrics=['accuracy']) 
model.fit(X, y_train_binary, epochs=300, validation_split=0.2, batch_size=10, verbose=0)
prediction_classes = model.predict_proba(t_X)
numpy.savetxt("C:/Users/Megaport/Desktop/foo.csv", prediction_classes, delimiter=",") 

I've tried so many parameters (learning rate, epochs, batch size, epsilon, add layer, less layer, different value of dropout) but the problem is the same: probs are not working.

the probabilities of the class are 1 or 0 and not a value between 1 and 0. This happen after 100 epoches.

data test with prediction after 100 epoches is like this:

**RESULT**   VALUE_A     VALUE_B     VALUE_C    **PRED_0    PRED_1**
   0           4          5          3           1          0
   0           7          4          5           0          1
   1           6          7          6           0          1
   1           2          3          4           0          1

What I'm looking for:

**RESULT**   VALUE_A     VALUE_B     VALUE_C    **PRED_0    PRED_1**
   0           4          5          3           0.65          0.35
   0           7          4          5           0.25          0.75
   1           6          7          6           0.20          0.80
   1           2          3          4           0.30          0.70

Plus, someone could please give me some advice looking at accuracy and loss graph?

$\endgroup$
0
$\begingroup$

Try predicting log-probabilities that would give you bigger seperation (i.e. values between -inf and 0) I Would clip these values (so that you dont take log from 0 which would be -inf but something like if 0 than log(0.0001) For example:

y_pred = estimator.predict_proba(X.drop(columns=self.drop_columns))
log_prob = lambda x: np.log(np.clip(x, 0.001, 0.999)).reshape(-1, 1)
y_pred = np.hstack([log_prob(y_pred[:, 0]), log_prob(y_pred[:, 1])])
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.