I'm trying to build a gesture recognition system for classifying ASL (American Sign Language) Gestures, so my input is supposed to be a sequence of frames either from a camera or a video file then it detects the sequence and maps it to it's corresponding class (sleep, help, eat, run, etc.)
The thing is I've already built a similar system but for static images (no motion included), it was useful for translating alphabets only in which building a CNN was a straight forward task, as the hand doesn't move so much and the data set structure was also manageable as I was using keras and maybe still intending to do so (every folder contained a set of images for a particular sign and the name of the folder is the class name of this sign ex: A, B, C,..)
My question here, how can I organize my data set to be able to input it into a RNN in keras and what certain functions should I use to effectively train my model and any necessary parameters, some people suggested using TimeDistributed class but I don't have a clear idea on how to use it for my favor, and take in account the input shape of every layer in the network.
also considering that my data set would consist of images, I'll probably need a convolutional layer, how would it be feasible to combine the conv layer into the LSTM one (I mean in terms of code).
For example I imagine my data set to be something like this
Folder named 'Run' contains 3 folders 1, 2 and 3, each folder corresponds to it's frame in the sequence
So Run_1 will contain some set of images for the first frame, Run_2 for the second frame and Run_3 for the third, my model's objective is to be trained with this sequence to output the word Run.