23
$\begingroup$

I want to train a deep learning model on a dataset containing around 3000 images. Since the dataset is huge, I want to use Google colab since it's GPU supported. How do I upload this full image folder into my notebook and use it?

Improve this question
$\endgroup$

6 Answers 6

Reset to default
14
$\begingroup$

Method 1 :

  1. zip the file
  2. Upload the zipped file, there is an Upload button under the Files Section.
  3. Unzip it using the command on colab : !unzip level_1_test.zip

Method 2 :

  1. upload the zip file to the google drive account.
  2. The only difference is in step 2 where in place of the GUI upload option you can run the google code_snippets to upload download your zip file from the google drive to Colab account .
  3. Unzip it using the command on colab : !unzip level_1_test.zip
Improve this answer
$\endgroup$
1
9
$\begingroup$

The best bet would be to upload the images as a zip file to your Google drive and then access it through Google Colab (GC)

  1. Zip the image folder
  2. Upload the zip file to your Google drive

  3. Turn to GC to authorise and mount your Google drive

    from google.colab import drive
drive.mount('/content/drive')

  4. Follow the link and paste the code to your GC notebook

  5. Unzip the file from GC

    !unzip -uq "/content/drive/My Drive/PATH_TO_ZIP" -d "/content/drive/My Drive/PATH_TO_OUTPUT"

  6. The files are now ready to use

Improve this answer
$\endgroup$
1
  • $\begingroup$ it is very slow method $\endgroup$
    – alex3465
    May 7, 2021 at 13:30
3
$\begingroup$

You don't need to upload them if you have a download link ...( it would be faster if you can upload them all as either ways you have to do so.. So its better to upload them first and then download them in your notebook every-time you run it)

If you have a download link then just this

! wget <Link>

Else upload then to your drive and then just use the following

from google.colab import files

uploaded = files.upload()

##files.upload returns a dictionary of the files which were uploaded. The 
##dictionary is keyed by the file name, the value is the data which was 
##uploaded.

for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name=fn, length=len(uploaded[fn]))
Improve this answer
$\endgroup$
0
0
$\begingroup$

I recommend you uploading a zip file containing your images to your drive and downloading the content from drive to Colab. Then you will be able to extract them. The code for uploading is here.

Improve this answer
$\endgroup$
0
$\begingroup$

You can upload stuff to Google Drive and then download it from there on Colab. I've written some utils for that - see this notebook.

As to how upload files to Google Drive, Media's suggestion is useful - upload zipped image folder.

Improve this answer
$\endgroup$
0
$\begingroup$

On Colab simply use:

!gdown --id file_id

For instance, if your Google drive share link for the file is:

https://drive.google.com/file/d/zz2Xs5Vriz6aF3V-Z22112yAj91c222fI1F/view?usp=sharing

use:

!gdown --id zz2Xs5Vriz6aF3V-Z22112yAj91c222fI1F
Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.