Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I want to build a classifier which uses multiple videos. I know video classifiers exist, but they rely on a single video.
The obvious solution is to create an ensemble classifier which uses the outputs of classifiers trained on a single video to provide multi-video classification. For example, training a MLP or XGBoost on the outputs of the pre-softmax layer.
While this is ok, I was wondering whether there are any more elegant/specialized end-to-end solutions for this?
One way to look at it is like multi-modality training, where all inputs are of the same modality - a practical example is to run a DNN on each video, and then concatenate the embeddings and run the rest of the network based on it. This is similar to the solution you describe but the training is unified and is done end to end.
P.S. Another obvious solution is to concatenate the videos and feed them as a single input to a general video classifier.
