# Change feature importance in a trained model

I am giving a toy example for describing a real world business problem. Let's say I am a publisher and I have some book stores to visit. By visiting those stores I will check whether they have sufficient stock of my books, they are visible on shelf etc. Now, I am training a model, to recommend me the stores to visit. I have, say, 20 features and some historical data which has a target variable which represents whether a store was visited or not (1/0). I trained a RandomForestClassifier model on that data and this is the feature importance I got.

feature_14, feature_2 are more important than feature_11. Now, assume that feature_11 is a feature with high business importance. As an example, let's say feature_11 is the number of books on shelf and I want to put more importance on it in deciding whether to visit a store. Is there any way to put more importance on this feature_11 than feature_14 or feature_2? From historical data, model has learned that it is third most importance variable but I want to make it a key deciding factor. Is there any way of doing it?

## 1 Answer

If I am understanding the real-world aspect of this correctly, it sounds like you are trying to decide on a set of sites to visit. Overall model "variable importances" are generally not a great approach here, and you want to try to find something from the predictive model itself related to the decision-of-interest.

You may want to consider using something different from the previous round of site-visits as the target variable. Instead, the modeling target should be an expected benefit that depends on a site visit. Then, after a model is built, you can use the results to generate a ranked list site-visits predicted to provide the largest benefit.

One approach would be:

1. Define some profitability or other benefit metric, that was available both before and after the site visits in your data were made.
2. If your system can model on a continuous variable, use the before-after difference directly as the target. Otherwise, bin the differences into t-shirt size classes (S/M/L/XL if you can run a multi-class classifer; S/L if you are limited to binary modeling)
3. Using site-visit as an input variable, build your random-forest model on your benefit-difference target.
4. Now run your cases through the predicted model, once with visited=No and once with visited=Yes.
5. Sort the results in order of biggest predicted difference between "visited" predictions. The top N sites that provide the biggest positive difference are the first N sites to visit.

If feature_11 is of business importance, i.e. it is related to profitability or some other success metric, it should fall naturally out of the analysis. However, a word of caution: For any kind of prediction based on past examples method to work, there needs to be some random variation between the variables. For example, if every case where feature_11 is large was also visited last year, then the model doesn't have anything to go on when feature_11=large/visited=No are suspect, and predictions assuming those conditions are suspect.