0
$\begingroup$

I have just started learning image processing and this is my first time working on video classification. I am trying to develop a model that recognizes hand gestures using the EgoGesture dataset(more info: http://www.nlpr.ia.ac.cn/iva/yfzhang/datasets/egogesture.html). This dataset consists of videos taken by 50 people in 6 different environments containing 83 gesture categories. Also, some of these gestures are dynamic. But I am using data from only 18 people because my memory is not enough. Since this dataset consists of videos, I take 10 equally distributed frames from each gesture video Like this: Gesture Frames

I resize each frame to be 128x128 pixels and normalize it. In total, I get 79320 frames from 7932 videos. I set the shape of X to (7932, 128, 128, 10, 3). Then I split this 80-20.

Here is my 3D CNN model code:

model = Sequential()

model.add(Conv3D(32, (3, 3, 3), activation='relu', input_shape=(128, 128, 10, 3)))
model.add(MaxPooling3D((2, 2, 2)))
model.add(Conv3D(64, (3, 3, 3), activation='relu'))
model.add(MaxPooling3D((2, 2, 2)))
model.add(Conv3D(64, (3, 3, 3), activation='relu', padding='same'))

model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))

optimizer = Adam(learning_rate=0.000001, beta_1=0.9, beta_2=0.999)

epochs = 30
batch_size = 10

model.compile(optimizer = optimizer , loss = "categorical_crossentropy", metrics=["accuracy"])

model.summary()

When I train this model, the training accuracy rate is around 0.04 and the validation accuracy rate is around 0.02.

I tried a more complex model by adding a few layers but there was no difference. I played with the hypermeters and again I could not achieve the desired success. I tried another model with CNN+LTSM and again I could not achieve a successful result.

Then I tried to use Depth videos (black and white segmented hands) instead of RGB videos in the dataset and both accuracy values increased to around 0.15 but again not enough.

It's obvious that I'm doing something wrong but I can't figure out what it is. Is my model wrong or maybe the data is not enough? I would be very grateful if you can help me. I will add more information if more information is needed.

Improve this question
$\endgroup$
2
  • $\begingroup$ Have you tried increasing the learning rate? The current learning rate seems very low given that the default learning rate for adam is 1e-4. $\endgroup$
    – Oxbowerce
    Sep 13 at 17:02
  • $\begingroup$ It was 0.001 at the start then I just decreased it over time. Maybe I try it again but don't think that's the problem. $\endgroup$
    – esyilmaz
    Sep 13 at 19:18

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.