2
$\begingroup$

My assumption is that my training set includes the test set, but I don't know how to change this.

from sklearn.model_selection import train_test_split
import sklearn.metrics as metrics

# dataframe to store model performances
scores=pd.DataFrame([],columns=['model', 'recall', 'f1', 'accuracy'])
from sklearn.tree import DecisionTreeClassifier

cv_scores_ac=[]
cv_scores_f1=[]    
cv_scores_re=[]            
for cv in range(1, 6):
    print ("Decision Tree - Iteration %i" % cv)
    X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=cv)

    tree=DecisionTreeClassifier(min_samples_split=20)
    tree.fit(X_train, y_train)
    y_pred=tree.predict(X_test)
    cv_scores_ac.append(metrics.accuracy_score(y_test, y_pred))
    cv_scores_f1.append(metrics.f1_score(y_test, y_pred))
    cv_scores_re.append(metrics.recall_score(y_test, y_pred))
$\endgroup$
4
  • $\begingroup$ How much is the length of X and Y? $\endgroup$ Commented May 20, 2020 at 16:36
  • $\begingroup$ My data set has 12 columns (incl. the target column), and 100,000 lines. does that answer it ? :) $\endgroup$
    – Sofia M
    Commented May 20, 2020 at 16:49
  • 1
    $\begingroup$ Why don't you test your hypothesis that the test data is included in the training data? $\endgroup$ Commented May 20, 2020 at 16:56
  • $\begingroup$ I meant check the length of X.Train, Y.Train, X.Test, Y.test. you probably ain't using the same data if you are calling.FIT on x_train,y_train.How are you splitting X and Y? Also check the confusion matrix, is only the accuracy high? what about precision , recall? Also, see if its over fitting? $\endgroup$ Commented May 20, 2020 at 16:57

2 Answers 2

2
$\begingroup$

Machine Learning is one of the few things where 99% is excellent and 100% is terrible.

Well, I cannot prove this because I don't have your data, but probably:

  • the test data is included in the training data.

To check this possibility, here's a hint:

print(X_test[X_test.isin(X_train)])

will print all the rows in X_test that appear in X_train. Can you think what to do with that information? :)

Or

  • you forgot to remove the target variable y from X.

To check this possibility, simply type:

print(X_train.columns)

and check whether one of the columns matches the name of the target variable.

$\endgroup$
4
  • $\begingroup$ I think the same, one of those - but how do I do this ? can you help with the piece of the code I have to change? $\endgroup$
    – Sofia M
    Commented May 20, 2020 at 17:10
  • $\begingroup$ sure, one moment $\endgroup$ Commented May 20, 2020 at 17:11
  • 1
    $\begingroup$ Solved! :) Thank you !!! $\endgroup$
    – Sofia M
    Commented May 20, 2020 at 20:48
  • $\begingroup$ If you think this solved your problem, please don't forget to press the green checkmark button at the left of my answer :-). Also, was the problem the first list item? I suspect that. $\endgroup$ Commented May 20, 2020 at 21:06
0
$\begingroup$

The most probably is data leakage, in this situation, you have the same values in train and test data(or linearly dependent). So please check out the input values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.