1
$\begingroup$

I want to do cross validation. So should i split my data into train and test with sklearn train_test_split and use cross validation like this:

cross_validate(model, X_train, y_train, scoring = roc_auc, cv = 5, n_jobs = -1)

Or should i make a function and split the data inside each fold differently like this:

def cross_val_evaluation(model):
    kf = KFold(n_splits=5)
    for train_index, test_index in kf.split(X,y):
        X_train, X_test = X.iloc[train_index, :], X.iloc[test_index, :]
        y_train, y_test = y.iloc[train_index], y.iloc[test_index]  
        y_pred_proba = model.predict_proba(X_test)
        cross_val_roc_auc_score = round(roc_auc_score(y_test,y_pred_proba[:,1]),3)
        cross_roc_auc_score_lst_test.append(cross_val_roc_auc_score_test)
    print(f'AUC Score Cross_Val Proba Test: {round(np.mean(cross_roc_auc_score_lst_test),3)}') 
$\endgroup$

1 Answer 1

1
$\begingroup$

First, split the dataset into train and test dataset. Then, cross-validate on the train dataset.

You should use the scikit-learn functions. Implementing the functionality yourself may introduce bugs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.