Feature Importance Scores Python

Question

I have a dataset having 7 attributes viz., time, C1, ... C7 pertaining to earth quake reports where each column/attribute represents a certain aspect of damage viz., power, sewer_and_water, shake_intensity, etc. Each of these attributes have a rating from 0 to 10, where 0 represents no damage and 10 represents maximum damage.

Since, some of the ratings in each attribute might not be trustworthy, hence I am trying to compute a weighted score as follows- (C1 * W1) + (C2 * W2) + ... + (C7 * W7); where Ci is ith attribute and Wi is the feature importance score for it.

In order to compute a weighted score of each row/data point, I am trying to train a classifier such as Random Forest, LightGBM or XGBoost which will give me a feature importance score for each attribute. However, since I don't know which attribute is supposed to be target, I used a brute force approach where I chose each attribute as target and trained a classifier to see which gave me the highest accuracy.

However, this approach has the problem that out of n attributes, I will only get feature importance score for (n - 1) attributes as the nth attribute is the target variable.

Can you suggest a way in which I can get the feature importance scores for each attribute in dataset.

Thanks!

Answer 1

Not all the algorithms provide you feature_importance, some of them give you coeff(weights W1..W7) values. So according to me, the way you are extracting feature_importance is correct.

However, this approach has the problem that out of n attributes, I will only get feature importance score for (n - 1) attributes as the nth attribute is the target variable.

Model gets trained based upon the features and target value. So you will get the feature_importance only for features and not for the target.

Using statistical approach you can also analyse the relationship between the various features and the target value.

As a side note:

• Make sure you normalized the whole features on the same scale for better and faster training.