Assume a binary classification problem, with $1$ denoted as a "bad" outcome, and $0$ as a "good" outcome. If it's relevant, in the sample there are significantly more bads than goods.
I'm trying to develop a classification model, where the desired outcome is the probability, rather than only the output class.
However, whatever variable/model combinations I try, the models are able to distinguish the bad cases quite nicely, but not the good cases. In other words, the distribution function of (smoothed) empirical outcomes vs the model estimated probabilities is not monotone, roughly having the shape of a tilted parabola (like letter $J$).
My question: What are the common strategies in order to reshape any model to focus the estimation on the good cases? If it is possible at all?
Intuitively, it seems that a possible strategy would be to define a custom metric for the minimizer, that would have asymmetric weights for good vs bad cases. I.e., in the case of penalized linear regression, the variable selection could then be skewed towards distinguishing goods instead of maximizing total AUC. But so far I'm unable to find any realized similar solutions. I guess, implementing something like that is not a trivial task..
Alternatively, is it possible to achieve this by transforming the input variables in a certain way?