Let's assume we perform a ML based classification in terms of some supervised ML approach. In view of a bias-variance trade-off, we decide to use the k-fold cross-validation strategy. For each fold we get a confusion matrix and a classification report ...eventually a ROC-curve. Thus we get k such results. One might characterize the overall score of each such fold by the mean-squared error (MSE). Thus, we get k times different MSE scores. One might take the average of the MSE's and find the mean score of the model based on the k-fold cross-validation, MSE_mean.
Here is the code that I run in the case of SVM:
from sklearn.model_selection import KFold
from sklearn.utils import shuffle
from sklearn.svm import SVC
from sklearn.metrics import classification_report, confusion_matrix
def SVM(X_train, X_test, y_train, y_test, kernelType):
svclassifier = SVC(kernel=kernelType)
svclassifier.fit(X_train, y_train)
y_pred = svclassifier.predict(X_test)
confusionMatrix = confusion_matrix(y_test,y_pred)
classificationReport = classification_report(y_test,y_pred)
return confusionMatrix, classificationReport
xall, yall = shuffle(x, y, random_state=21)
kf = KFold(n_splits=5)
from imblearn.over_sampling import SMOTE
for train_index, test_index in kf.split(x):
yall.iloc[train], yall.iloc[test]
x_train = xall.iloc[train_index,:]
y_train = yall.iloc[train_index]
x_test = xall.iloc[test_index,:]
y_test = yall.iloc[test_index]
# smote
smote = SMOTE(sampling_strategy='minority')
x_train_sm, y_train_sm = smote.fit_sample(x_train, y_train)
x_train_sm, y_train_sm = shuffle(x_train_sm, y_train_sm, random_state=21)
#confusionMatrixSVM, classificationReportSVM
confusionMatrixSVM, classificationReportSVM = SVM(x_train_sm, x_test, y_train_sm, y_test, 'rbf')
print(confusionMatrixSVM)
print(classificationReportSVM)
My questions: Since we do ML in order to make predictions, we want to have a single model at the end of the k-fold cross-validation process and not separate k models. How do we get a single model at the end ? That model should in principle provide a score MSE = MSE_mean. I need that model for making predictions. Can you suggest some code for getting a single model after the k-fold cross-validation ?
Many thanks.