OpenCV isn't processing all the images in a directory

Question

I'm really trying to apply CLAHE to a directory of about 700 images. However, once I manage to get it running, for some reason the code stops before all the images are processed.
When I run it on Google Colab, it stops halfway (so at image# 355). Out of frustration I downloaded the image dataset to my local machine and tried to run it on a local Jupyter notebook, but even then it stops after 100 images. What's going on?

Here's my code:

import cv2 as cv
import shutil
import os

source = r"/All classes copy/0/"
destination = r"/temp dir/0/"
filelist = os.listdir(r"/All classes copy/0/")

clahe = cv.createCLAHE()

for x in filelist:
    img = cv.imread(x,0)
    cl1 = clahe.apply(img) 
    cv.imwrite(os.path.join(destination, x),cl1)
    print(os.path.join(source, x), '>>>', os.path.join(destination, x))
    filelist0.pop()

Here's the error, which comes up only after a couple hundred images have been CLAHE'd:

---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
Input In [28], in <cell line: 11>()
     12 img = cv.imread(x,0)
     13 cl1 = clahe.apply(img)
---> 14 cv.imwrite(os.path.join(r"/temp dir/0/", x),cl1)
     15 print(os.path.join(source, x), '>>>', os.path.join(destination, x))
     16 filelist0.pop()

error: OpenCV(4.6.0) /Users/runner/work/opencv-python/opencv-python/opencv/modules/imgcodecs/src/loadsave.cpp:801: error: (-215:Assertion failed) !_img.empty() in function 'imwrite'

However I know for sure that these images do exist! It processes more images in Colab than locally. Is this a stability issue with OpenCV? Or is it that I'm running out of RAM?

Answer 1

I figured it out. Actually, there were a few issues that stopped this code from executing smoothly.

1. Make sure you are in the 'source' directory for this to work, because the first line of the code (img = cv.imread(x,0)) does not have an explicit path, and so is calling on any files in the local directory with the name x
2. Make sure there are no non-image files in the source directory
3. Because Colab can seem a bit erratic at times, it may help to create a new (and differently-named) filelist variable with each loop (if you are doing this over several directories)
4. And finally, follow the guidance code to the T! I first placed the clahe = cv.createCLAHE() instantiation outside the loop as I thought it only needed to be created once. But doing that stops the loop early, for some reason. You have to include it in the loop for the loop to be executed throughout the whole source directory. The final code would then be this:

import cv2 as cv
import shutil
import os

source = r'/content/drive/MyDrive/Colab Notebooks/All classes/0/'
destination = r'/content/drive/MyDrive/Colab Notebooks/CLAHE/0/'
filenames0 = os.listdir(r'/content/drive/MyDrive/Colab Notebooks/All classes/0/')


for x in filenames0:
  img = cv.imread(x,0)
  clahe = cv.createCLAHE()
  cl1 = clahe.apply(img)
  cv.imwrite(os.path.join(destination, x),cl1)
  print(os.path.join(source, x), '>>>', os.path.join(destination, x))
```