Related to TrackNet, a CNN for tracking tennis balls on TV tennis matches, the Arxiv paper mentions it is scalable, ie. the input can be any number of frames concatenated rather than the three they used. So I tried to concatenate 11 frames and adjusted the input layer dimension:
#changed from 9 to 33 for 11 frames input imgs_input = Input(shape=(33,input_height,input_width))
But now when I try to load a weights file that comes with the open source code, I am getting an error:
Traceback (most recent call last): File "predict_video.py", line 55, in <module> m.load_weights( save_weights_path ) File "/usr/local/lib/python2.7/dist-packages/keras/engine/network.py", line 1166, in load_weights f, self.layers, reshape=reshape) File "/usr/local/lib/python2.7/dist-packages/keras/engine/saving.py", line 1058, in load_weights_from_hdf5_group K.batch_set_value(weight_value_tuples) File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 2465, in batch_set_value assign_op = x.assign(assign_placeholder) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variables.py", line 1952, in assign name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/state_ops.py", line 227, in assign validate_shape=validate_shape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_state_ops.py", line 66, in assign use_locking=use_locking, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func return func(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3616, in create_op op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2027, in __init__ control_input_ops) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1867, in _create_c_op raise ValueError(str(e)) ValueError: Dimension 0 in both shapes must be equal, but are 3 and 64. Shapes are [3,3,33,64] and [64,9,3,3]. for 'Assign' (op: 'Assign') with input shapes: [3,3,33,64], [64,9,3,3].
The actual input for the original CNN is 3 video frames of height 360, width 640 and the code looks like this:
imgs_input = Input(shape=(9,input_height,input_width))
And the model is instantiated like this:
m = modelFN( n_classes , input_height=height, input_width=width )
n_classes is a command line argument with default value of
For 11 frames, I tried instantiating the 3 frames model, loading the weights and then instantiating the 11 frames model and tried to used
old_model.get_weights() specified in this answer:
So the model and weights loading snippet looks like this:
#load TrackNet model modelFN = Models.TrackNet.TrackNet m = modelFN( n_classes , input_height=height, input_width=width ) m.compile(loss='categorical_crossentropy', optimizer= 'adadelta' , metrics=['accuracy']) #load and save from same path m.set_weights( save_weights_path ) #load TrackNet 11 frames model and transfer weights model11 = Models.TrackNet11.TrackNet11 m11 = model11(n_classes, input_height=height, input_width=width) m11.compile(loss='categorical_crossentropy', optimizer='adadelta', metrics=['accuracy']) m11.load_weights(m.get_weights)
The full code is available at the link below
I tried the Stackoverflow answer and tried to used
None for the channel dimension because using
33 gave me an error saying
dim2 is different ie.
[3,3,9,64] but now I am getting:
ValueError: The channel dimension of the inputs should be defined. Found `None`.
So the channel dimension has to be defined.
I am going to try this: datasciencestackexchange answer
But this means that the weights from inputs to first conv2d layer will not be the pretrained ones?
Anyways, I did try it but was unable to get any output, ie. it did not track the tennis ball at all and I am pretty sure there are no other errors in the code but will double check. If anyone has a easy solution that would be appreciated.
My attempt at converting from 3 frames concatenated input to 11 frames can be seen at the following link in files
predict_video11.py. In the
Models folder you will see
TrackNet.py for 3 frames and
TrackNet11.py for 11. There is also a python 3 version that I converted to from the original python 2 version using
py2to3 that works and comes with
requirementspy3.txt assuming you have the correct version of tensorflow installed for your machine (cpu or gpu with cuda, cudnn).
Arxiv paper link: Arxiv TrackNet