1
$\begingroup$

I have dataset of 17-20 videos (5 minutes) each video representrepresents a different class (around 4 classes). I can'tcannot get more video because this was provided by the company. I'm planingI am planning to split the video path and then apply video capture to get the video frame as an image of each video then apply a CNN for classification.

I know my data is small (couldthis could result in overfitoverfitting) how can I handle thatthis when I can'tcannot increase the number of videos ?


 train_data, extra_data = train_test_split(video_path, train_size=0.9, shuffle=False) #90

validation_data, test_data = train_test_split(extra_data, test_size=0.2, shuffle=False #10

$\endgroup$
2
  • 1
    $\begingroup$ Since you (at least kind of) treat each frame as a separate image, you might have a lot more data than you think. $\endgroup$
    – Dave
    Commented Nov 12, 2022 at 19:47
  • $\begingroup$ Please don't make more work for other people by vandalizing your posts. By posting on the Stack Exchange (SE) network, you've granted a non-revocable right, under the CC BY-SA 4.0 license for SE to distribute that content. By SE policy, any vandalism will be reverted. If you want to know more about deleting a post, consider taking a look at: How does deleting work? $\endgroup$
    – Glorfindel
    Commented Feb 12, 2023 at 12:06

1 Answer 1

1
$\begingroup$

You have limited videos which you will be converting to images to train the model further. There is always data augmentation which you can do but again keeping in mind that it should not be too much.

To avoid over-fitting look at the training loss and generalization loss after you have done data augmentation. If the gap between those two is too big you will face over-fitting. If the value of training loss is too much you will face under fitting.

Data scarcity is always bottle-neck due to which most of the models never go into production.

$\endgroup$
1
  • $\begingroup$ Since I'm doing video classification looking at my dataset if the min size of my data is 17 video what should the min size for the validation test be ? Since I have 5 category I was thinking that each category must be in the training and testing data however I was left with 1 video for validation would that be appropriate? (11 train, 5 test data, 1 validation data) $\endgroup$
    – user142691
    Commented Nov 14, 2022 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.