Why are only 3-4 features important in my random forest?

Question

I am running a random forest regression with Python's Scikit-Learn, code's below (X - features, y - to be predicted).

# Splitting the dataset into the Training set and Test set
from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 1)
    
# Scaling
from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
X_train = sc_X.fit_transform(X_train)
X_test  = sc_X.transform(X_test)

# Random forest
from sklearn.ensemble import RandomForestRegressor
rf =RandomForestRegressor(max_depth=2, n_estimators = 100, random_state=0)
rf = rf.fit(X_train,y_train)
pred_train = rf.predict(X_train)
pred_test = rf.predict(X_test)

I am running this code for randomly sampled 100k dataset, that has 60+ features. Each time when I check feature importance I get 3 to 4 variables as important (with one of them holding over 80% of importance), and others' importance is set to 0. It is not reasonable to me that only these are important for prediction and the rest is rubbish.

var_num = X_train.shape[1]
plt.barh(range(var_num), rf.feature_importances_, align='center')
plt.yticks(np.arange(var_num), variable_names)
plt.xlabel('Variable Importance')
plt.ylabel('Variable')
plt.show()

Is it possible that I am missing something? That some other parameters needed to be defined? Could this be caused by a high correlation between variables themselves? Or is it really that the rest of my features are useless..?

Answer 1

RandomForestRegressor has a parameter called max_features, which is the number of features to consider when determining the optimal split. You haven't explicitly specified this, so Python will use the default (auto) and consider all features.

Given that your trees are very shallow and you are considering all of the features to split, it would not surprise me that the strongest 3-4 are consistently popping up (the bagging process in random forests will cause some variation within this).

Decreasing max_features and/or increasing max_depth may yield a greater variety of "important" features.