Is my model overfitting?

Question

I'm currently building my first model with sklearn to predict whether a customer will renew a subscription. I'm using a random forest because I've heard that they are robust to overfitting. The confusion matrix generated on the results of my test set (25%) seem to imply that the model is working very well, but the tree I'm generating is massive. The size of the tree alone, especially considering I'm only using 12 variables, makes it seem like the model is creating too many complex relationships and thus must be overfitting. If the model is overfitting how can I adjust my code to fix this? Here is the code I used to tune hyper parameters and fit the model.

pipe = Pipeline([('classifier', RandomForestClassifier())])


param_grid = [
{'classifier' : [RandomForestClassifier()],
'classifier__n_estimators' : list(range(200,500,10)),
'classifier__max_features' : list(range(6,32,5))
}
]

clf = GridSearchCV(pipe, param_grid = param_grid, cv = 5, verbose=True, n_jobs=-1)

predictions = clf.best_estimator_.predict(x_test)
cm = metrics.confusion_matrix(y_test, predictions)
plt.figure(figsize=(9,9))

sns.heatmap(cm, annot=True, fmt=".3f", linewidths=.5, square = True, cmap = 'Blues_r');
plt.ylabel('Actual Values')
plt.xlabel('Predicted Values')
plt.title('Renewal Model Confusion Matrix')

Here is the confusion matrix generated, as well as an image of the tree I'm generating and the code used to generate it.

from sklearn.tree import export_graphviz
import graphviz
import os
os.environ["PATH"] += os.pathsep + 'C:/Users/rich/Anaconda3/envs/32bitpy/Library/bin/graphviz/'

d = export_graphviz(clf.best_estimator_[0][1], out_file='rf.dot', 
            rounded = True, proportion = False, 
            precision = 2, filled = True)

os.environ["PATH"] += os.pathsep + 'C:/GraphViz/bin'

graph = graphviz.Source(d)
graph

Answer 1

If your model is overfitting then you can use Dropout, L1 & L2 Regularization. It helps to improve your model's accuracy.

Answer 2

There are lots of experiment u can try. One might not decide the overfitting and underfitting based on the confusion matrix completely.

1. Deal with imbalance data : First prepare a smooth sampling of your data where u need to keep thing in mind to take uniform number of sample in each class and run it and evaluate its accuracy. Because sometime due to imbalance data it seems it gives good accuracy but it have been biased model.
2. Feature Engineering : If playing with random forest than try to find out gini index or entropy value of each and every feature (in your case is 12). Where u will see which feature is prominent each class. That's how u might neglect some of the features and run new experiment and check out results.