Video Segmentation With 5 Lines of Code

Semantic and instance segmentation of videos.

The first version of PixelLib is built to perform Image Segmentation using few lines of Code. I am excited to announce that the newly released version of PixelLib supports Video Segmentation with five lines of code.

If you have not read the article on Image Segmentation With PixelLib, click here .

Install tensorflow with:

Install PixelLib with:

pip3 install pixellib — upgrade

Semantic Segmentation of Videos:

Segmentation of videos with pascal voc model:

We shall explain each line of code below.

import pixellibfrom pixellib.semantic import semantic_segmentationsegment_video = semantic_segmentation()

We imported in the class for performing semantic segmentation and created an instance of the class.

segment_video.load_pascalvoc_model("deeplabv3_xception_tf_dim_ordering_tf_kernels.h5")

We loaded the xception model trained on pascal voc dataset to perform semantic segmentation and it can be downloaded from here .

segment_video.process_video_pascalvoc("video_path",  overlay = True, frames_per_second= 15, output_video_name="path_to_output_video")

We called the function to perform segmentation on the video file.

It takes the following parameters:-

video_path : the path to the video file we want to segment.
frames_per_second: this is the parameter used to set the number of frames per second for the saved video file. In this case it is set to 15 i.e the saved video file will have 15 frames per second.
output_video_name: the name of the saved segmented video . The output video will be saved in your current working directory.

sample_video1

Output Video

This is the saved segmented video in your current working directory.

Semantic Segmentation of Live Camera.

We can use the same model to perform semantic segmentation on camera. This can be done by including few modifications to the code used to process a video file.

import cv2capture = cv2.VideoCapture(0)

We imported cv2 and included the code to capture camera’s frames.

segment_video.process_camera_pascalvoc(capture,  overlay = True, frames_per_second= 15, output_video_name="output_video.mp4", show_frames= True,frame_name= "frame", check_fps = True)

In the code for performing segmentation, we replaced the video’s filepath to capture i.e we are processing a stream of frames captured by the camera instead of a video file.We added extra parameters for the purpose of showing the camera’s frames:

show_frames: this parameter handles showing of segmented camera’s frames.
frame_name: this is the name given to the shown camera’s frame.

check_fps: You may want to check the number of frames processed per second, just set the parameter check_fps to true . It will print out the number of frames per seconds. In this case it is 30 frames per second .

Awesome! 30 frames per second is great for Real Time Segmentation of camera’s feed.

A demo showing the output of pixelLib’s semantic segmentation of camera’s feeds using pascal voc model.

Good work! It was able to successfully segment me and the plastic bottle in front of me.

Instance Segmentation of Videos:

import pixellib
from pixellib.instance import instance_segmentation segment_video = instance_segmentation()

We imported in the class for performing instance segmentation and created an instance of the class.

segment_video.load_model("mask_rcnn_coco.h5")

We loaded the maskrcnn model trained on coco dataset to perform instance segmentation and it can be downloaded from here .

segment_video.process_video("video_path", frames_per_second = 20, output_video_name = "output_video.mp4")

We called the function to perform instance segmentation on the video file.

It takes the following parameters:-

video_path: this is the path to the video file we want to segment.
frames_per_second: this is the parameter used to set the number of frames per second for the saved video file. In this case it is set to 15 i.e the saved video file will have 15 frames per second.
output_video_name: this is the name of the saved segmented video . The output video will be saved in your current working directory.

sample_video2

We can implement instance segmentation with bounding boxes by setting the parameter show_bboxes to true .

Output video

Instance Segmentation of Live Camera.

We can perform instance segmentation on camera’s frames. This can be done by including few modifications to the code used to process a video file.

import cv2 capture = cv2.VideoCapture(0)

We imported cv2 and included the code to capture camera’s frames.

segment_video.process_camera(capture, show_bboxes = True, frames_per_second = 15, output_video_name = "output_video.mp4", show_frames = True, frame_name = "frame", check_fps = True)

In the code for performing segmentation, we replaced the video’s filepath to capture i.e we are processing a stream of frames captured by the camera instead of a video file. We added extra parameters for the purpose of showing the camera frames:

show_frames: this parameter handles the showing of segmented camera’s frames.
frame_name: this is the name given to the shown camera’s frame.

check_fps: You may want to check the number of frames processed, just set the parameter check_fps is true . It will print out the number of frames processed per second. In this case it is 30 frames per second.

A demo showing the output of pixelLib’s instance segmentation of camera’s feeds using Mask-RCNN.