3
$\begingroup$

I have been made a model using both Decision Tree and Random Forest. But, when I tried to test the model on the same DataFrame the output is different. How is this possible?

The data file from my repo:

#This is the function to help me preparing the dataframe
def process_df_for_ml(df):
    """
    Process a dataframe for model training/prediction use.

    Returns X/y tensors.
    """

    df = df.copy()
    # Map salary to 0,1,2
    df.salary = df.salary.map({"low": 0, "medium": 1, "high": 2})
    # dropping left and sales X for the df, y for the left
    X = df.drop(["left", "sales"], axis=1)
    y = df["left"]
    return (X, y)

I used the decision tree:

from sklearn import tree
from sklearn.model_selection import train_test_split
# Train a decision tree.
X, y = process_df_for_ml(df)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0, stratify=y)
clftree = tree.DecisionTreeClassifier(max_depth=3)
clftree.fit(X_train, y_train)

With test_score: 0.96. Afterwards, I test this DecisionTree to the same df and the output I got is [424 rows x 11 columns]

and then I tried to use Random Forest algorithm

X, y = process_df_for_ml(df)

from sklearn.model_selection import train_test_split
# implementing train-test-split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=0, stratify=y)

from sklearn.ensemble import RandomForestClassifier
# random forest model creation
rfc = RandomForestClassifier()
rfc.fit(X_train,y_train)
# predictions
rfc_predict = rfc.predict(X_test)

With test_score: 0.99. Afterwards, I test this RandomForest to the same df and the output I got is [11 rows x 11 columns].

How is this possible? Here is the link to my works: DecisionTree and RandomForest

$\endgroup$
  • 2
    $\begingroup$ Even though the ideas in both models are similar, at the end of the day they are different models, no reason for them to give same the same result. $\endgroup$ – serali Nov 2 '19 at 10:18
  • $\begingroup$ @serali which models are usually superior, is it models from decision tree or random forest? And in this case which models should I believe? $\endgroup$ – ebuzz168 Nov 2 '19 at 10:42
  • 2
    $\begingroup$ I don't think it is possible to say which is superior without experimentation. Here is another post discussing pros and cons of each model in detail: stats.stackexchange.com/questions/285834/… $\endgroup$ – serali Nov 2 '19 at 10:45
  • $\begingroup$ @serali so basically, my works are already right but the different given results need more study to answer the why it's different? $\endgroup$ – ebuzz168 Nov 2 '19 at 12:04
  • $\begingroup$ I don't understand what you mean when you say "the output I got is <shape>"; what output? $\endgroup$ – Ben Reiniger Nov 5 '19 at 20:19
1
$\begingroup$

1. Data Preparation

By default, train_test_split will assume your train:test split to be 75:25 if you don't declare otherwise for either the test_size or train_size parameters. See here.

For your decision tree, you don't declare this and so you test on 25% of the data, whereas you explicitly state in your data preparation for the random forest algorithm that the train size should be 33%.

Correcting for this ensures a fair comparison of the two algorithms in this scenario. I have assumed you wish to use test_size=0.33.

2. Model Differences

The random forest classifier is an ensemble method. As its name implies, this method generates a 'forest' of many decision tree classifiers that each train on a different subset of samples and features from the training data. The n_estimators, max_samples and max_features parameters control this. See here. The results of each tree are then averaged with the aim of providing a more robust estimator.

In order for the two classifiers to produce the same output, I can set the same random_state and max_depth whilst allowing all features to be used (since by default all samples are used in every tree generated) in the random forest:

tree.DecisionTreeClassifier(random_state=0, max_depth=3)
ensemble.RandomForestClassifier(random_state=0, max_depth=3, max_features=None)

Both now achieve an identical test accuracy of 0.9551 because both classifiers generate the same identical tree.

Removing max_features=None allows the random forest classifier to randomly select features for each subset trained upon by each respective tree in the random forest:

ensemble.RandomForestClassifier(random_state=0, max_depth=3)

Resultantly, the random forest classifier now achieves a test accuracy of 0.9121.

Hopefully, this explains why you were getting different results for the decision tree and random forest classifiers.

| improve this answer | |
$\endgroup$
1
$\begingroup$

Your df row count is 14,999

Your test data is 33% ~ 4950

So, your y_predict should be (4950,1) i.e. a binary prediction for all test rows.

This is what I am getting when running your code which you have posted here.

import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/bhaskoro-muthohar/DataScienceLearning/master/HR_comma_sep.csv')

print(df.size)
# Map salary to 0,1,2
df.salary = df.salary.map({"low": 0, "medium": 1, "high": 2})
# dropping left and sales X for the df, y for the left
X = df.drop(["left", "sales"], axis=1)
y = df["left"]

#splitting the train and test sets
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=0, stratify=y)
# Train a decision tree.
from sklearn.tree import DecisionTreeClassifier
clftree = DecisionTreeClassifier(max_depth=3)
clftree.fit(X_train, y_train)
y_pred = clftree.predict(X_test) 
print(y_pred.size)
from sklearn.ensemble import RandomForestClassifier
# random forest model creation
rfc = RandomForestClassifier()
rfc.fit(X_train,y_train)
# predictions
rfc_predict = rfc.predict(X_test)
print(rfc_predict.size)
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.