1
$\begingroup$

The code I used was taken from shapes.py, balloon.py, inspect_balloon_data.ipynb but mostly from train_shapes.ipynb in the Matterport repo because I read from some posts on Stackoverflow that adapting balloon.py and the ipynb notebooks there are the fastest way to get up and running on your own datasets annotated using the VGG Annotation tool (version 1 as version 2 won't work with the code). Link to VGG Annotation tool: VGG annotation tool I made some changes to balloon.py so that I can use the circle bounding circle rather than a polygon:

At the bottom of def load_balloon():

self.add_image(
                "tennisball",
                image_id=a['filename'],  # use file name as a unique image id
                path=image_path,
                width=width, height=height, #come from right above
                circles=circles)

and in def load_mask():

#convert circles to a bitmap mask of circle
#[(x,y) of center, radius]
info = self.image_info[image_id] #jpg num
mask = np.zeros([info["height"], info["width"], 
     len(info["circles"])],dtype=np.uint8)
for i, p in enumerate(info["circles"]):
    rr, cc = skimage.draw.circle(p["cx"],p["cy"], p["r"])
    mask[rr, cc] = 1

it looks like the tennis ball is detected but the mask is placed in the wrong location.

FYI my repo is here with the notebook and balloon.py my repo

Edit: In def load_balloon() in balloon.py I made some other changes to go along with the other code pasted here:

 # Add images
 for a in annotations:
 # Get the x, y coordinaets of points of the polygons that make up
 # the outline of each object instance. These are stores in the
 # shape_attributes (see json format above)
 # The if condition is needed to support VIA versions 1.x and 2.x.
            if type(a['regions']) is dict:
                #polygons = [r['shape_attributes'] for r in a['regions'].values()]
                circles = [r['shape_attributes'] for r in a['regions'].values()]
            else:
                #polygons = [r['shape_attributes'] for r in a['regions']]
                circles = [r['shape_attributes'] for r in a['regions']] 

In the meantime, this github repo changes rectangle annotations in XML to the format required by Matterport Mask R-CNN code (ie. all points x and y that make up the perimeter of polygons): xml to json

Edit: I was able to do some transfer learning on Colab and my result:

tracking the color yellow at least, I hope

The mask and bounding box are in the bottom left near the score. At least it's sort of trying to bound the yellow part of the score I'm hoping. This was with a training set of 21 images and validation of 10 images which is way below the recommended 100 to 1000 for transfer learning. I just wanted to see that the code executes on Colab with minor adjustments (must mount Google Drive). Full ipynb notebook is here: tennisball-tracker

Edit: I think I'm coming to the conclusion that this task is impossible for mask-r-cnn for tennis balls on TV broadcasts without a high FPS camera because the ball is so blurry to the point that it's nearly invisible in some frames. Ie. This frame right above shows Rafa finishing his serve motion so the ball is near its top speed in a tennis match and I cannot spot the ball. Thus I cannot make a good training dataset from raw video frames shown on regular TV broadcasts. Please refer to this other question if you are also trying to do the same task: concatenation of consecutive video frames

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.