Return to Answer

It could be the way that you encode categorical variables.

If you do One Hot Encoding (dummy) each encoded feature will only have two possible values [0,1]. Binary variables normally have less importance in Decision trees given how it is computed the feature weight.

Let's say per example that you are trying to predict the condition of a patient at the hospital Alive==0, Dead ==1. Imagine that you have a feature is called Head_Shot[0,1], that is really rare, it only appears a few times in the dataset.

The linear model will assign a lot of weight to this coefficient since it is crucial for the target variable. If this happens the rest of the features has no meaning.

For the decision tree, it won't be so important since it calculates the importance of a feature weighted time the number of times it appears.

I am assuming you are doing one-hot encoding. With other techniques, it will be different. And I also assume the way that you calculate the feature importance. So this is far from a scientific answer.

Sorry for the example, it is a bit drastic but I believe it makes the point.

It could be the way that you encode categorical variables.

If you do One Hot Encoding (dummy) each encoded feature will only have two possible values [0,1]. Binary variables normally have less importance in Decision trees given how it is computed the feature weight.

Let's say per example, that you are trying to predict the condition of a patient at the hospital Alive==0, Dead ==1. Imagine that you have a feature is called Head_Shot[0,1], that is really rare, it only appears a few times in the dataset.

The linear model will assign a lot of weight to this coefficient since it is crucial for the target variable. If this happens the rest of the features has no meaning.

For the decision tree, it could do a split in just one of the tree and since it calculates the importance of a feature weighted time the number of times it appears it wouldn´t such a relevant weight.

I am assuming you are doing one-hot encoding. With other techniques, it will be different. And I also assume the way that you calculate the feature importance. So this is far from a scientific answer.

Continuous variables can have more importance in decision trees because each tree can do several splits along its way.

Sorry for the example, it is a bit drastic but I believe it makes the point.

It could be the way that you encode categorical variables.

If you do One Hot Encoding (dummy) each encoded feature will only have two possible values [0,1]. Binary variables normally have less importance in Decision trees given how it is computed the feature weight.

Let's say per example that you are trying to predict the condition of a patient at the hospital Alive==0, Dead ==1. Imagine that you have a feature is called Head_Shot, that is really rare, it only appears a few times in the dataset.

The linear model will assign a lot of weight to this coefficient since it is crucial for the target variable. If this happens the rest has no meaning.

For the decision tree, it won't be so important since it calculates the importance of a feature weighted time the number of times it appears.

I am assuming you are doing one-hot encoding. With other techniques, it will be different. And I also assume the way that you calculate the feature importance. So this far from a scientific answer.

Sorry for the example, it is a bit drastic but I believe it makes the point.

It could be the way that you encode categorical variables.

If you do One Hot Encoding (dummy) each encoded feature will only have two possible values [0,1]. Binary variables normally have less importance in Decision trees given how it is computed the feature weight.

Let's say per example that you are trying to predict the condition of a patient at the hospital Alive==0, Dead ==1. Imagine that you have a feature is called Head_Shot[0,1], that is really rare, it only appears a few times in the dataset.

The linear model will assign a lot of weight to this coefficient since it is crucial for the target variable. If this happens the rest of the features has no meaning.

For the decision tree, it won't be so important since it calculates the importance of a feature weighted time the number of times it appears.

I am assuming you are doing one-hot encoding. With other techniques, it will be different. And I also assume the way that you calculate the feature importance. So this is far from a scientific answer.

Sorry for the example, it is a bit drastic but I believe it makes the point.

It could be the way that you encode categorical variables.

If you do One Hot Encoding (dummy) each encoded feature will only have two possible values [0,1]. Binary variables normally have less importance in Decision trees given how it is computed the feature weight.

Let's say per example that you are trying to predict the condition of a patient at the hospital Alive==0, Dead ==1. Imagine that you have a feature is called Head_Shot, that is really rare, it only appears a few times in the dataset.

The linear model will assign a lot of weight to this coefficient since it is crucial for the target variable. If this happens the rest has no meaning.

For the decision tree, it won't be so important since it calculates the importance of a feature weighted time the number of times it appears.

I am assuming you are doing one hot encoding. With other techniques it will be different.

Sorry for the example, it is a bit drastic but I believe it makes the point.

It could be the way that you encode categorical variables.

If you do One Hot Encoding (dummy) each encoded feature will only have two possible values [0,1]. Binary variables normally have less importance in Decision trees given how it is computed the feature weight.

Let's say per example that you are trying to predict the condition of a patient at the hospital Alive==0, Dead ==1. Imagine that you have a feature is called Head_Shot, that is really rare, it only appears a few times in the dataset.

The linear model will assign a lot of weight to this coefficient since it is crucial for the target variable. If this happens the rest has no meaning.

For the decision tree, it won't be so important since it calculates the importance of a feature weighted time the number of times it appears.

I am assuming you are doing one-hot encoding. With other techniques, it will be different. And I also assume the way that you calculate the feature importance. So this far from a scientific answer.

Sorry for the example, it is a bit drastic but I believe it makes the point.