Techniques for increase random forest classifier accuracy

Question

I build basic model for random forest for predict a class. below mention code which i used.

from sklearn.preprocessing import StandardScaler
ss2= StandardScaler()
newdf_std2=pd.DataFrame(ss2.fit_transform(df2),columns=df2.columns)

from sklearn.model_selection import train_test_split
x_train,x_test,y_train,y_test = train_test_split(df2,y2,test_size = 0.3, random_state = 0)

from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(n_estimators=100, max_depth=2, random_state=0)
clf.fit(x_train,y_train)

clf.score(x_test,y_test)

output accuracy of the model 0.758266818700114

I got 75% accuracy but i want increase accuracy furthermore, so what are thing i want to do this code, for increase accuracy? or is there any classification techniques better than random forest please suggest? this data set have lot of outliers and missing values before build model i removed all missing values and outliers

Answer 1

• You can try XGBoost or LightGBM, they often perform better than Random Forest
• Try do not remove missing values, complex ensemble models such as RF and GBM treats it well, may be you lost some useful information doing so, especially if you have large percent of your data missing in some features
• Try do not remove outliers, sometimes it's better to have it in your data
• Try to increase n_estimators and max_depth, may be your trees not deep enough to catch all data properties

Answer 2

I guessing why are you scaling the feature before the model. as far as a know feature scaling is applied only on those algo's where a distance is calculated (e.g. k-means). Maybe this could be a improving for you accuracy model. I totaly agree with @CrazyElf , switching to XGBOOST or other algo's could increase the accuracy. then last but not least, missing values are preatty important (if they are a lot maybe there's no need to drop them, but better consider them means to catch some insight)