I am running a random forest regression with Python's Scikit-Learn, code's below (X - features, y - to be predicted).
# Splitting the dataset into the Training set and Test set from sklearn.cross_validation import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 1) # Scaling from sklearn.preprocessing import StandardScaler sc_X = StandardScaler() X_train = sc_X.fit_transform(X_train) X_test = sc_X.transform(X_test) # Random forest from sklearn.ensemble import RandomForestRegressor rf =RandomForestRegressor(max_depth=2, n_estimators = 100, random_state=0) rf = rf.fit(X_train,y_train) pred_train = rf.predict(X_train) pred_test = rf.predict(X_test)
I am running this code for randomly sampled 100k dataset, that has 60+ features. Each time when I check feature importance I get 3 to 4 variables as important (with one of them holding over 80% of importance), and others' importance is set to 0. It is not reasonable to me that only these are important for prediction and the rest is rubbish.
var_num = X_train.shape plt.barh(range(var_num), rf.feature_importances_, align='center') plt.yticks(np.arange(var_num), variable_names) plt.xlabel('Variable Importance') plt.ylabel('Variable') plt.show()
Is it possible that I am missing something? That some other parameters needed to be defined? Could this be caused by a high correlation between variables themselves? Or is it really that the rest of my features are useless..?