1
$\begingroup$

How do I arrange the image dataset in CNN? Should I put each image category in a separate folder? Or all of them in the same folder? Should the image name be the category name?

I would like to see an example for an image dataset (other than MNIST).

Thank you.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
4
$\begingroup$

Directory structure like in dogscats/.(atleast I kept it this way)

dogscats

    |-- train

          |-- cats

                |-- catpic0, catpic1, …

          |-- dogs/

                |-- dogpic0, dogpic1, …

    |-- valid

          |-- cats

                |-- catpic0+x, catpic1+x, …

          |-- dogs

                |-- dogpic0+x, dogpic1+x, …

    |-- test

           |-- catpic0+x+y, catpic1+x+y, dogpic0+x+y, dogpic1+x+y

Becareful With the naming of the files also..

Also note that you will need a mapping of image names and classes as well like in a CSV or something...

Also there is no globally accepted directory structure, it completely depends on the API you will use...

What is a good train/validation/test split?(depends on your dataset size)

  • can do $80/20$ (train/validation)

if you have or are creating a 'test' split, use for (train/validation/test):

  • can do $80/15/5$
  • can do $70/20/10$
  • can do $60/20/20$

Remembering that sole aim is to generalize eventually on test sets...

Improve this answer
$\endgroup$
5
  • $\begingroup$ How can I map the image name and class in a CSV file? Do you mean create an CSV file and then fill the first column with the image name and the second column with the class name? $\endgroup$
    – N.IT
    Jul 11, 2018 at 13:33
  • 1
    $\begingroup$ Yes and no both, generally it's provided or you have to make a custom one... With the probability of the former being more... $\endgroup$
    – Aditya
    Jul 11, 2018 at 14:56
  • 1
    $\begingroup$ If I can recall correctly, I did write such a script here as an answer, you can search a bit or write from scratch $\endgroup$
    – Aditya
    Jul 11, 2018 at 15:13
  • $\begingroup$ I want to build the dataset from scratch $\endgroup$
    – N.IT
    Jul 11, 2018 at 16:26
  • $\begingroup$ Yes it's possible, there's a GitHub repo here's the link github.com/hardikvasa/google-images-download $\endgroup$
    – Aditya
    Jul 12, 2018 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.