As context, I am relatively new to the world of machine learning and I am attempting a project with a goal of classifying plays in an NBA game. My inputs are a sequence of 40 frames from each play in an NBA game and my labels are 11 all-encompassing classifications for a given play.
The plan is to take each sequence of frames and pass each frame into a CNN to extract a set of features. Then each sequence of features from a given video would be passed onto an RNN.
I am currently using Keras for most of my implementation and I chose to use a VGG16 model for my CNN. Here is some of the relevant code below:
video = keras.Input(shape = (None, 255, 255, 3), name = 'video')
cnn = keras.applications.VGG16(include_top=False, weights = None, input_shape=
(255,255,3), pooling = 'avg', classes=11)
cnn.trainable = True
My question is - would it still be beneficial for me to initialize the weights of the VGG16 ConvNet to 'imagenet' if my goal is to classify video clips of NBA games? If so, why? If not, how can I train the VGG16 ConvNet to get my own set of weights and then how can I insert them into this function? I have had little luck finding any tutorials where someone included their own set of weights when using the VGG16 model.
I apologize if my questions seem naive but I would really appreciate any help in clearing this up.