1
$\begingroup$

How do I arrange the image dataset in CNN? Should I put each image category in a separate folder? Or all of them in the same folder? Should the image name be the category name?

I would like to see an example for an image dataset (other than MNIST).

Thank you.

$\endgroup$
3
$\begingroup$

Directory structure like in dogscats/.(atleast I kept it this way)

dogscats

    |-- train

          |-- cats

                |-- catpic0, catpic1, …

          |-- dogs/

                |-- dogpic0, dogpic1, …

    |-- valid

          |-- cats

                |-- catpic0+x, catpic1+x, …

          |-- dogs

                |-- dogpic0+x, dogpic1+x, …

    |-- test

           |-- catpic0+x+y, catpic1+x+y, dogpic0+x+y, dogpic1+x+y

Becareful With the naming of the files also..

Also note that you will need a mapping of image names and classes as well like in a CSV or something...

Also there is no globally accepted directory structure, it completely depends on the API you will use...

What is a good train/validation/test split?(depends on your dataset size)

  • can do $80/20$ (train/validation)

if you have or are creating a 'test' split, use for (train/validation/test):

  • can do $80/15/5$
  • can do $70/20/10$
  • can do $60/20/20$

Remembering that sole aim is to generalize eventually on test sets...

$\endgroup$
  • $\begingroup$ How can I map the image name and class in a CSV file? Do you mean create an CSV file and then fill the first column with the image name and the second column with the class name? $\endgroup$ – N.IT Jul 11 '18 at 13:33
  • 1
    $\begingroup$ Yes and no both, generally it's provided or you have to make a custom one... With the probability of the former being more... $\endgroup$ – Aditya Jul 11 '18 at 14:56
  • 1
    $\begingroup$ If I can recall correctly, I did write such a script here as an answer, you can search a bit or write from scratch $\endgroup$ – Aditya Jul 11 '18 at 15:13
  • $\begingroup$ I want to build the dataset from scratch $\endgroup$ – N.IT Jul 11 '18 at 16:26
  • $\begingroup$ Yes it's possible, there's a GitHub repo here's the link github.com/hardikvasa/google-images-download $\endgroup$ – Aditya Jul 12 '18 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.