I am working on a Human Activity Recognition on videos using deep learning and want to train my Neural Network (ConvLSTM & LRCN) on my custom dataset. The issue that I am facing is that my dataset is more than 50 GB and I don't have that much memory to load it into the memory directly(16 GB RAM and Nvidia P2000 with 5 GB VRAM). I read about the TensorFlow
flow_from_directory which is used to effectively load the dataset, but my problem is that I would like to load multiple images for single label.
For example, a video might have $n$ number of frames in it and I extract the required frames from it for training my NN on it, for instance a 30 sec video of walking label might have 2000 frames and I want to get every 20th frames (20,40,60...2000).
Now, how do I deal with sequence of these frames while using TensorFlow
flow_from_directory? I have 40 Videos of each label (40 Walking, 40 Running, 40 jogging and so on) how do I tell TensorFlow that these $n$ frames belong to this single video(such as walking label) and how do I send them at once to the model to train on it?
As far as I know, the default behavior of
flow_from_directory is that if we have 4 folder (cat, dog, zebra and fish) it go to each folder and gets each photo from it and assign the label same as the label name. But in my case I have for instance 3 folders (Walking, Running and Jogging) and in each folder I have Nested folders (Walking Video 1, Walking Video 2 and so on) and in each folder there are n number of frames belonging to that video label. How do I deal with that?
Nested folders Structures
Bold represents a folder
Walking (Walking Video 1 (n frames belong to video 1), Walking Video 2 (n frames belong to video 2) and so on for all the walking videos)
Running (Running Video 1 (n frames belong to video 1), Running Video 2 (n frames belong to video 2) and so on for all the Running videos)
and so on for all the labels