2
$\begingroup$

I have developed a model in Keras that works perfectly when reading data stored locally. However, I now want to take advantage of Google Cloud Platform's GPUs for training the model. I have set up the GPU on GCP and am working in a Jupyter notebook. I have moved my images to Google Cloud Storage.

My question is:

How can I access these images (specifically the directories - training, validation, test) directly from Cloud Storage using the Keras' flow_from_directory method of the ImageDataGenerator class?

here's my directory structure in Google Cloud Storage (GCS):

mybucketname/
      class_1/
          img001.jpg
          img002.jpg
          ...
      class_2/
          img001.jpg
          img002.jpg
          ...
      class_3/
          img001.jpg 
          img002.jpg
          ...
$\endgroup$
2
  • 1
    $\begingroup$ I've found that copying the files directly from Cloud Storage to the VM works via import os, sys os.system('gsutil cp -r gs://mybucketname/ .') but that's not the elegant solution I was hoping for… $\endgroup$ – Ryan Chase Apr 20 '18 at 0:06
  • $\begingroup$ Have you found a better solution? I am currently searching for one and I have only found some workarounds. $\endgroup$ – moondra Feb 17 '19 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.