0
$\begingroup$

How to deal with videos where the frame sizes are not the same frame to frame?

For example this video moves up and down and when it does, the video part of the screen has a different amount of pixels vertically.

How to deal with different frame sizes in a CNN?

$\endgroup$
0

1 Answer 1

1
$\begingroup$

In your example, you are seeing the effect of a stabilisation algorithm applied to otherwise jerky movement - and the effect is a moving "window" inside the video frame.

The actual video file has a fixed frame size. As far as I know there are no video formats that switch size frame to frame. You may have different areas of focus to worry about, but should never need to worry about frame size switching on you mid video. You can rely on your video processing library to present a consistent frame size for the whole of a video.

If you find that the border of the view inside the video frame interferes with the performance of your CNN once trained, then you could deal with it in a few ways. A couple of simple approaches might be:

1) Crop out a centre portion that always reliably has content in, and use just that portion for your input features.

2) Detect and flood-fill the black border with the mean pixel value, using an image/video library (like OpenCV) so that edges around it cause less activity in your CNN.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.