My assumption is that my training set includes the test set, but I don't know how to change this.

from sklearn.model_selection import train_test_split
import sklearn.metrics as metrics

# dataframe to store model performances
scores=pd.DataFrame([],columns=['model', 'recall', 'f1', 'accuracy'])
from sklearn.tree import DecisionTreeClassifier

for cv in range(1, 6):
    print ("Decision Tree - Iteration %i" % cv)
    X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=cv)

    tree.fit(X_train, y_train)
    cv_scores_ac.append(metrics.accuracy_score(y_test, y_pred))
    cv_scores_f1.append(metrics.f1_score(y_test, y_pred))
    cv_scores_re.append(metrics.recall_score(y_test, y_pred))
  • $\begingroup$ How much is the length of X and Y? $\endgroup$ – BlackCurrant May 20 at 16:36
  • $\begingroup$ My data set has 12 columns (incl. the target column), and 100,000 lines. does that answer it ? :) $\endgroup$ – Sofia M May 20 at 16:49
  • 1
    $\begingroup$ Why don't you test your hypothesis that the test data is included in the training data? $\endgroup$ – Guillermo Mosse May 20 at 16:56
  • $\begingroup$ I meant check the length of X.Train, Y.Train, X.Test, Y.test. you probably ain't using the same data if you are calling.FIT on x_train,y_train.How are you splitting X and Y? Also check the confusion matrix, is only the accuracy high? what about precision , recall? Also, see if its over fitting? $\endgroup$ – BlackCurrant May 20 at 16:57

Machine Learning is one of the few things where 99% is excellent and 100% is terrible.

Well, I cannot prove this because I don't have your data, but probably:

  • the test data is included in the training data.

To check this possibility, here's a hint:


will print all the rows in X_test that appear in X_train. Can you think what to do with that information? :)


  • you forgot to remove the target variable y from X.

To check this possibility, simply type:


and check whether one of the columns matches the name of the target variable.

| improve this answer | |
  • $\begingroup$ I think the same, one of those - but how do I do this ? can you help with the piece of the code I have to change? $\endgroup$ – Sofia M May 20 at 17:10
  • $\begingroup$ sure, one moment $\endgroup$ – Guillermo Mosse May 20 at 17:11
  • 1
    $\begingroup$ Solved! :) Thank you !!! $\endgroup$ – Sofia M May 20 at 20:48
  • $\begingroup$ If you think this solved your problem, please don't forget to press the green checkmark button at the left of my answer :-). Also, was the problem the first list item? I suspect that. $\endgroup$ – Guillermo Mosse May 20 at 21:06

The most probably is data leakage, in this situation, you have the same values in train and test data(or linearly dependent). So please check out the input values.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.