What is the best way to calculate score based on input features?

Question

Lets say we have certain products. We also have certain input features regarding these products like inventory, sales, price, cost etc. based on these features we want to score these parts. The score should reflect whether it is in shortage or not. Shortage is when there is high demand (Sales) and low inventory/supply. Score of 100 means it is in shortage. Score of 1 means it is not in shortage.

Right now i use a random forest model and use the probability from the model as the score. However this leads to a lot of products having score between 0-10 and 90-100. Very few products have score between 10-90.

Is there a better way to do this so that the scores are more distributed?

Answer 1

I assume that currently you're training a binary classification model, right?

You could try training a regression model which predicts the score between 0 and 100. It would be better to have some examples in your training data which are between 10 and 90, because that would make the model learn the distribution of the scores. But even if you don't have this kind of instances, it's possible (not sure though) that the model will predict instances in this range in some cases.

Answer 2

Regarding the scores between 10-90, I'd think the training data could be such that there are very few samples in that set.

This looks like a regression problem, try training an XGB regressor over your training data. The implementation for which can be found in sklearn documentation. It would help if your training samples are of good quality i.e. Sufficient samples from every range.