I am building a Tennis stroke classification system using CNN.

I assume each stroke contains 3 steps/classes ('Ready', 'Impact', 'Finish'). I want to train a model which will predict whether the input video contains these steps/classes in it.

I have tried training 3 models for each step as binary classification.

Example of one step model classes:

1 - ready  
0 - not-ready(other incorrect steps). 

But this method failed since there are more features in 'not-ready' class. I got only 4% accuracy.

Can anyone help me to find a solution for this problem.

  • $\begingroup$ why not instead of 3 binary classification use a single multi-class classification? $\endgroup$
    – WBM
    Apr 12, 2021 at 13:36
  • $\begingroup$ @WBM According to my knowledge, when you input a video, output will be a single class. i.e. Ready or Impact or Finish. But in my case, I want predict whether all three classes are in a single video. $\endgroup$
    – Kavee
    Apr 12, 2021 at 14:06

1 Answer 1


Given that you have only 3 classes and that they closely depend on each other, I think it's worth trying a multiclass setting as WBM said. The idea is to label each video using the full combination of actions, since the maximum number of combinations is 2^3 = 8:

  • R-I-F
  • R-I
  • R-F
  • R
  • I-F
  • I
  • F
  • none

Probably some combinations of actions are impossible, so the number of classes is likely less than 8. Why this is a reasonable approach:

  • The setup is exactly the same, i.e. you can use the same labels and the predictions can be used the same way as in your multi-label approach
  • This is a "joint model", i.e. a model which learns everything together and therefore can exploit fine-grained distinction between classes (e.g. between R-I-F and R-I).

However note that this kind of method may require more data, in particular it needs to have enough instances for each class.

  • $\begingroup$ Thanks for replying. But when I train the model, I split the whole dataset into frames and feed for the model to train. Is there a way to do it without splitting into frames so that the model gains awareness of the content of the whole video? For an example, to identify an actions in the video which spans across multiple frames. Then the above mentioned answer is feasible to do. $\endgroup$
    – Kavee
    Apr 15, 2021 at 4:01
  • $\begingroup$ @Kavee I'm not expert in image/video but I'm pretty sure that the model would indeed perform better if it can exploit the order of the frames. This means that you need a model which considers the input as an ordered sequence. I know that there are such sequence classification models but I don't know the state of the art model(s) for a video classification task. $\endgroup$
    – Erwan
    Apr 15, 2021 at 11:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.