Calculate a ranking function from classification features

Question

I am using 3 features (x1, x2, x3) for binary classification. All my feature values are in 0 to 1 range (unit range).

I obtained how important each feature was in classification as follows (i.e. feature importance)

x1 --> 0.1
x2 --> 0.5
x3 --> 0.7

It is clear that feature 3 (x3) contributes the most, x2 the second and x1 the least in classification.

I also performed correlation analysis to check if my features are positively or negatively correlated with the target (y) as follows.

x1 --> positively correlated
x2 --> positively correlated
x3 --> negatively correlated

I am wondering if it is possible to convert my classification features into a ranking function using feature importance and correlation.

For instance, my suggestion looks as follows.

ranking_score = 0.1*x1 + 0.5*x2 + 0.7*(1/x3)

The reason for using (1/x3) in the above equation is because it is negatively correlated with the target (y). Please let me know if my ranking_score equation is statistically correct? If not, please let me know your suggestions.

EDIT: Why ranking is important to me?

My features are related to employee details (x1, x2, x3). At first I used these 3 features to classify efficient and 'inefficient' employees. Now, I want to rank the efficient employees based on these 3 features. The above equation I proposed is to facilitate this task.

I am happy to provide more details if needed.

Answer 1

Generally speaking moving from a classification space to a ranking space is not straight forward. In classification problems there is no meaning to order between labels. This means that your suggested equation might not represent the order between labels at all.

This is somewhat dependent on the feature space and classification algorithm. Some classification algorithms(tree-based for example) don't use the concept of distance in their search for best fit. The separation created from the fit(which is used for classification) does not include how far the boundaries are from one another.

Some classification algorithms (logistic regressions, SVM, others) do have this distance feature, and this can be translated to the probability of an instance belonging to each class. Using this probability as a ranking mechanism (instead of how asking how efficient, asking what is the probability of being efficient) might make sense.

Answer 2

Why don't you try getting weights using ahp and then do ranking using topsis or vekor?