Object detection in live video stream with Chrysalis Cloud

TL;DR

This post is an adaptation of a existing pre-trained object detection API ML model form Googles Colab Tutorial to run in the cloud. As the original Object Detection API Demo walks you step by step through the process of using a pre-trained model to detect objects in an image, here we adapt that concept to run on a remote live video stream. We don’t dive into the details of Tensorflow 2, inference nor any other details of training a custom model but rather demonstrate the ease of adapting an existing ML model to run in the Chrysalis Cloud.

About Tensorflow 2 Object Detection API

The TensorFlow Object Detection API is an open source framework built on top of TensorFlow that makes it easy to construct, train and deploy object detection models. The pre-trained models have been re-implemented using Keras layers and the weights have been saved in the TF2 checkpoint format.

In this post we use ssd_mobilenet_v1_coco_2017_11_17 (Common Object in Context) pre-trained model.

Using Chrysalis cloud to detect objects in real-time video stream

This example is assuming you have a GPU capable graphics card. By default it assumes:

• cudatoolkit=10.1
• cudnn=7.6.5

Modify environment.yml to fit to your environment.

If you need some additional information as of what your GPU environment is here are some useful commands for Linux OS:

Check GPU card information:

nvidia-smi

Check cuda version:

nvcc --version

or

cat /usr/local/cuda/version.txt

Check cudnn version:

cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
or
cat /usr/include/cudnn.h | grep CUDNN_MAJOR -A 2

Prerequisites

Creating an endpoint on Chrysalis Cloud

For creating an endpoint you’ll need a developer account on Chrysalis Cloud.

Once logged in create your first RTMP Stream. It should take about a minute or less. Chrysalis cloud gives out a free trial for a month for one camera.

Install OBS Studio

In this example we will use OBS studio for development. You could also develop with OpenCV VideoCapture and later move onto Chrysalis Cloud. You will see from the code example how easy we can transition from local solution to cloud solution utilizing Chrysalis Cloud SDK.

Read this if you need  more information how to stream live video to Chrysalis Cloud from OBS Studio.

Alternative streaming methods:

• Read here how to stream from Mac
• Read here how to stream from Ubuntu

Install Anaconda

Since Chrysalis Cloud SDK is dealing with somewhat complex dependencies the easiest and recommended way is to use anaconda environment. Anaconda also comes in handy later when we need something even more complicated such as GPU acceleration or different versions of Tensorflow.

Find how to install here

Python Code

Clone the development repository:

git clone https://github.com/chryscloud/chryscloud-ai-examples.git

Create and activate conda environment:

cd object-detection
conda env create -f environment.yml
conda activate chrysobject

Tensorflow model setup

After creating and activating conda environment clone the tensorflow models:

git clone --depth 1 https://github.com/tensorflow/models

Compile the protos for object detection:

cd models/research
protoc object_detection/protos/*.proto --python_out=.
cp object_detection/packages/tf2/setup.py .
python -m pip install .
cd ../..
cp -r models/research/object_detection object_detection

Run

Get Chrysalis Cloud SDK streaming credentials for your connected camera and export to environment variables:

export chrys_port=1111
export chrys_host=url.at.chrysvideo.com
export chrys_password=mypassword
export chrys_cert=pathtomycertificate.cer

python object_detection.py

Resources

• Traning custom object detector – Tensorflow 2
• Extensive object detection API tutorial with Tensorflow 2
• More about COCO dataset