How can I load the custom data in the data loader (pytorch)

Question

Note: I have extracted the frame for all videos and save it in the folder with the same name of video

train_data, class, video ---> These are folders

img --> these are jpg files, so each class have many videos, I extracted the image for each video and save it to the folder with the name of video from which the frames are extracted.

Directory of my dataset is something like this;

Total extracted images for each video = 28

Total classes are = 101

Total Videos are = 10619

Total Images are = 301169

Temporal Length = 16

Temporal stride = 4

for each video ==> It will read first 16 images, then it will leave next 4 image and read from 5th one till 20, by leaving next four image it will again read from 9th image to 24, and then in last till 28 for each video.

Total number of sample will be 4 for each video ==> [16, 20, 24, 28] (28 extracted frames)

each sample contain 16 frames with 112x112x3 shape size.

total number of sample for all classes = num_sample_for_each_video * total videos = 4 * 10619 = 42142 (approximately because the sample can be 3 in certain videos)

This way, it will form the shape for training data ==> [42142, 16, 112, 112, 3] it will form the shape for label data ==> [42142, ]

Can Anyone tell me how can I load it on DataLoader in PyTorch?

data format is something like this;

[[img1_filename,img2_filename...,img16_filename],label1], [[img1_filename,img2_filename...,img16_filename],label2],...]

[[[16 images (0-16)], label1]], [[16 images (4,20)], label2], [[16 images (9,24)], label3], [[16 images (13,28)], label3]]

I write this code,

X_train = []
        y_train = []
        for data in tqdm(data):  # Loop over every batch
            # Load image (X)
            x = data[0]
            y = data[1]
            temp_data_list = []
            for img in x:
                try:
                    img = cv2.imread(img)
                    # apply any kind of preprocessing here
                    # img = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
                    # img = self.preprocess_image(img)
                    temp_data_list.append(img)

                except Exception as e:
                    print(e)
                    print('error reading file: ', img)
                    # Read label (y)
            # label = label_names[y]
            # Add example to arrays
            X_train.append(temp_data_list)
            y_train.append(y)

Answer 1

The data loading behaviour that you describe does not seem to be covered by any of the default Dataset classes so you'd have to create a custom one. The general structure of a Dataset in pytorch that can be used in a DataLoader is a class that inherits from Dataset and has two methods, __len__ and __getitem__ to get the number of samples and retrieving a sample from the dataset respectively. This would look something like this:

from pathlib import Path
from torch.utils.data import Dataset

class CustomDataset(Dataset):
    def __init__(self, directory):
        self.directory = directory
        self.images = Path(self.directory).glob("**/*.jpg")

    def __len__(self):
        return len(self.images)

    def __getitem__(self, index):
        # implement your custom logic that loads and
        # transforms the images as needed
        pass

You would have to define the logic you describe in the __getitem__ method (and maybe also the line which retrieves the image paths since you mention that multiple images combined are one sample). Once you have defined all the logic needed in your custom dataset you can using this dataset in the dataloader.