Let me start by saying my machine learning experience is... dangerous at this stage. I'm still a beginner.
I have a binary classification data set of about 100 000 records. 10% of the records are positive and the rest obviously negative. Thus a highly skewed dataset.
It is extremely important to maximize the positive (true positive) prediction accuracy (recall) at the expense of negative (true negative) prediction accuracy . Thus, I would rather have an overall 70% accuracy if positive accuracy is 90%+ compared to a low positive accuracy and high overall accuracy.
You can already see the issue here. Training the below algorithm obviously optimizes loss for the entire dataset. Thus, priority is given to the negative records which consist of 90% of the dataset. Thus, the overall data set accuracy is high, but the true positive accuracy (recall) is horrible.
model = keras.Sequential()
model.add(layers.Dense(128, activation='relu', input_dim=35))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
One idea would be to try and change the sigmoid threshold to less than 0.5 to try and give preference to recall. But to begin with I have no idea how to do this or if it is even a valid method. Any advice will be appreciated
