3
$\begingroup$

I am using Keras to create a CNN model, and I would to use K-fold cross-validation to train the dataset.

The dataset contains images and I am using flow_from_directory function.

Do you have any idea how to use K-fold cross-validation in Keras to create a CNN model??

$\endgroup$
4
  • $\begingroup$ Very expensive, this is why we use a validation set. $\endgroup$ Sep 24, 2018 at 6:33
  • $\begingroup$ Do you mean Cross-validation method is very expensive? $\endgroup$
    – N.IT
    Sep 24, 2018 at 12:09
  • $\begingroup$ Yes cross validation is considered expensive for dnn , because you have to do the training , testing k times $\endgroup$ Sep 24, 2018 at 14:52
  • 1
    $\begingroup$ I want to do cross validation, even if it is expensive $\endgroup$
    – N.IT
    Oct 2, 2018 at 5:02

2 Answers 2

0
$\begingroup$

It is very common to use sklearn for cross validation.

The most known methods are KFold and cross_val_score imported from sklearn.model_selection.

You can then use tf.keras.wrappers.scikit_learn.KerasClassifier to implement the scikit-learn classifier with Keras model.

$\endgroup$
0
$\begingroup$

K-fold cross-validation "consists of splitting the available data into K partitions (typically K = 4 or 5), instantiating K identical models, and training each one on K – 1 partitions while evaluating on the remaining partition" [Deep Learning with Python by François Chollet].

On the other hand, flow_from_directory takes the path to a directory & generates batches of augmented data [source: TensorFlow documentation]

Since the whole dataset is not accessible at the same time when using flow_from_directory, the required conditions for the K-fold cross-validation method are not met. So, K-fold cross-validation and flow_from_directory are incompatible. That being said, there might be other variants of K-fold cross-validation applying the same concept to each batch (as large as possible), but I have not heard of such an extension.

Why should it not be a problem in the first place?

Scenario A | small dataset: K-fold cross-validation is beneficial when one has so few data points that the validation set would end up being very small. In such cases, the whole dataset can be loaded off the disk and be divided into multiple partitions, e.g., using tf.data.Dataset.from_tensor_slices.

Scenario B | large dataset: In such cases, it may not be feasible to load the whole dataset at the same time (due to insufficient memory). Generating batches of data can resolve this issue. The K-fold cross-validation may not be even advantageous here due to having a large amount of data points (unless there are many classes with a low number of data points leading to a large-size dataset, and simultaneously an insufficient number of data points per class).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.