0
$\begingroup$

Let me start by saying my machine learning experience is... dangerous at this stage. I'm still a beginner.

I have a binary classification data set of about 100 000 records. 10% of the records are positive and the rest obviously negative. Thus a highly skewed dataset.

It is extremely important to maximize the positive (true positive) prediction accuracy (recall) at the expense of negative (true negative) prediction accuracy . Thus, I would rather have an overall 70% accuracy if positive accuracy is 90%+ compared to a low positive accuracy and high overall accuracy.

You can already see the issue here. Training the below algorithm obviously optimizes loss for the entire dataset. Thus, priority is given to the negative records which consist of 90% of the dataset. Thus, the overall data set accuracy is high, but the true positive accuracy (recall) is horrible.

model = keras.Sequential()
model.add(layers.Dense(128, activation='relu', input_dim=35))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

One idea would be to try and change the sigmoid threshold to less than 0.5 to try and give preference to recall. But to begin with I have no idea how to do this or if it is even a valid method. Any advice will be appreciated

$\endgroup$
3

1 Answer 1

0
$\begingroup$

This is the kind of solution that I was looking for:

https://github.com/huanglau/Keras-Weighted-Binary-Cross-Entropy/blob/master/DynCrossEntropy.py

$\endgroup$
1
  • $\begingroup$ As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    May 17, 2022 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.