Soccer Player Detection in Overhead Images using RetinaNet

Computer Vision for Sports

Detecting soccer players and ball with RetinaNet.

Straight-to-the-point guide to building a computer vision model to detect players and ball in overhead camera images.

We will be using the RetinaNet model as describe in the Focal Loss for Dense Object Detection paper by Tsung-Yi Lin , Priya Goyal , Ross Girshick , Kaiming He , Piotr Dollár .

We will also use Fizyr’s awesome implementation of keras-retinanet , applying it to Haizaha’s Soccer Player and Ball Detection free dataset.

Requirements:

First, make sure you have TensorFlow and Keras installed.

If you need GPU support, follow the instructions on tensorflow.org .

Next, you will need to clone, build, and install the keras-retinanet project.

The Data

After you’ve installed all the dependencies, the next step is to download the labeled images .

Pre-trained Weights

I‘ve made my training weights publicly available. Here is how you can download them if you want to run the model right away.

Imports

We’ll need to import TensorFlow, Keras, keras_retinanet, along with a few more modules.

Utilities

• Viewing annotations:

We will use the following view_annotations function to display images with associated bounding box annotations.

If you run the following command:

You should get an image like below:

Output of view_annotations()

• Loading a model:

• Running detection on an image with a model

Inference

Now that we’ve created helper functions, all we need to do is call them to load a model and run it on an image.

Load the model:

Using the model to detect players and ball:

Run_detection takes the model we created using the weights ‘pre-trained/resnet50_csv_last_inference.h5,’ and runs it on the ‘soccer/images/frame_0100.jpg’

You should get something like this:

Output of run_detection()

Training

To train your model, all you need to do is execute the train.py script:

Model training should start, and you should see something similar to this:

Screenshot: training progress in the terminal

Monitoring Training Progress

TensorBoard provides a neat web interface for monitoring your model learning. To run it, you just need to run the following code in the terminal:

tensorboard --logdir logs/fit

You want to make sure logs/fit points to the logs directory of

Now you can on your browser, go to the follow URL:

http://you_ir_address:6006

Screenshot: TensorBoard

Converting the training model

After you’ve trained the model, you will need to convert the training model into an inference model, before you can use it to detect object.

Here is how you convert a training model to an inference model:

Run the newly trained model

Now you can load the newly trained model and run detection again:

You should be expecting the following output from the code above:

Output of run_detection() after re-training the model

Optimization: Detecting Small Objects is Challenging

We’ve made impressive progress in the few years on the tasks of object detection. Despite that, today’s models are still struggling with detecting small objects within an image compared to detecting large objects.

Small Objects Detection , on PapersWithCode.com

RetineNet uses a Feature Pyramid Network (FPN) in its first stage to identify object on different scales. You can play with different scales and ratios values to help the network better identify small objects.

What you’ll need to do is create a config.ini for you model:

[anchor_parameters] 
sizes   = 32 64 128 256 512 
strides = 8 16 32 64 128 
ratios  = 0.5 1 2 3 
scales  = .5 1.2 1.6

When you train the model, make sure you include the config file:

python keras-retinanet/keras_retinanet/bin/train.py --config config.ini csv soccer/train.csv soccer/labels.csv --val-annotations soccer/valid.csv

Project Repository

All the code for this project is freely available on GitHub:

https://github.com/ramarlina/retinanet-player-ball-detection