We often use the classification threshold that maximizes the F1 score, if we don't have a prior cost function of false positives and false negatives.
This balances the desire for precision and recall. If either one is 0 the F1 score is 0; and if we have a perfect classification the F1 score is 1.
On the other hand I'm hard pressed to find a scientific justification to maximize F1 in general, or a business problem where F1 is the thing we need to maximize.
F1 is not symmetric. If we have an 60/40 binary distribution and choose the 40% class as the positive class, and we classify everything as positive, we get 100% recall and 40% precision for F1 score of 0.4. (F1=0 if we classify everything negative). If we choose the 60% class as positive, and classify everything positive, we get F1 score of 0.6.
Why not use mutual information, which minimizes the surprise in the prediction vs. actual?
When we estimate probabilities by minimizing log loss, we are also minimizing K-L divergence or entropy or surprise in the information theory sense. If a 50/50 probability prediction contains 1 bit of entropy or surprise, minimizing the log loss minimizes the number of bits of entropy or surprise in our predictions vs. actual.
And in information theory, if we don't have probabilities and want to measure the information transmitted by a noisy signal, we use mutual information.
And mutual information is symmetric, and 0 for a signal of all 1s or 0s. Predicting all 1s or 0s in some sense gives no information about the ground truth of the response variable.
In a way, a prediction is like a noisy channel from the present to the future, and an information theory concept like mutual information seems like a well-grounded criterion to choose the classification threshold, and F1 seems arbitrary.
Is there any reason why F1 is preferred over mutual information?