We’ve published previous posts on the birth of Chrysalis Cloud and how the platform works. In this article, we compare the process of (1) streaming, (2) analyzing, and (3) storing video using AWS Kinesis Video Streams vs. the Chrysalis Cloud. Read on to learn how you can save up to 70% on your cloud costs, while also reducing latency, and speeding up development/ deployment timelines.
Keep reading for the longer story:
We know the headlines: “Companies surprised by AWS bills,” “Cloud cost control becoming a leading issue for businesses.” AWS Cloud services for streaming, analyzing and storing video are expensive and can be very complicated to use. At Chrysalis Cloud, our goal is to make this process far simpler and cheaper, enabling new use cases for video and more advanced ML and AI in the cloud.
First, the basics of what we are comparing. Stream processing in the cloud has three phases:
- Ingestion (getting the data from the camera to the cloud),
- Processing (doing computation such as video analytics), and
- Storage/ Retrieval (keeping the stream for later use and accessing it when needed).
We are going to walk through each of these phases, comparing the process using AWS Kinesis Video Streams vs. the process using Chrysalis Cloud.
We’ll start with Ingestion.
Kinesis Ingestion: Even though Kinesis is a “streaming video service,” for many camera types it’s fairly difficult to get video data from the camera into a Kinesis stream.
For example, simply purchasing a “Kinesis stream” (which is pricey at ~$11/month for 10 records/second, 10KB per record) won’t get your data from the camera into the purchased Kinesis stream. The $11 Kinesis stream will only buffer and manage data; to actually send data into Kinesis, you need additional Amazon services. For example, if your camera uses RTSP to stream data off of it, you’ll need to run a separate Docker container, which means you’ll need an additional virtual machine. The same goes if you want to have the camera “push” data via RTMP.
You can read about how complicated this is in this forum on the AWS site. Amazon offers Kinesis SDK that allows you to push video directly from the camera, but this requires you to have the ability to run programs on the camera. If you want to use any off-the-shelf IP camera, this isn’t an option. Another huge overhead here is that once an EC2 instance is set up to handle video, the EC2 instance needs to be continuously monitored to ensure that the instance hasn’t crashed, stopped, or run out of resources.
Chrysalis Ingestion: Chrysalis ingestion process is much, much simpler, and cheaper than the equivalent Kinesis experience.
Setting up ingestion with Chrysalis is just a one-step process. Using the Chrysalis Web Portal (CWP), you simply request that a new Chrysalis Node be created on the cloud provider of your choice. The CWP (1) allocates a new virtual machine (VM) on your cloud provider, or if possible, re-uses an existing VM and (2) installs, starts, and continually monitors a Docker container on that VM to enable stream data ingestion. After creation, you get a simple URL for the Chrysalis Node and you can start streaming immediately.
There are already endpoints set up for common protocols such as RTMP. You simply set up your camera to publish to mynode.chryscloud.com/rtmp (exactly like what one would do to broadcast to Facebook Live or YouTube Live). For RTSP, and other pull-based methods, you can use the CWP to instruct the node to pull data from your camera into the Chrysalis Node.
Cost-wise, Chrysalis Nodes are very lightweight. Comparing directly against Kinesis:
Now let’s discuss video processing.
Kinesis Processing: In the AWS world, the user has several options for performing compute jobs on Kinesis streams. Jobs need to be done somewhere that can perform computation, which means either on another leased virtual machine (such as EC2) or using a serverless job-by-job platform such as Lambda.
Using Lambda, the system flow diagram with processing data is the same as before, except now we need to add a new service. This adds complexity and latency- a subtle, but important point. In the Kinesis model, your compute job runs on a different machine than where your data lives. This means that every time we need data, we have to first transfer it to the Lambda job.
This adds latency for transfer times, bandwidth costs, and complexity to the system. If there is a failure in the system, is it the Lambda job that failed? Did the transfer fail? Did the Kinesis stream have a problem? This type of multi-failure-point architecture can be devastatingly hard to debug.
Chrysalis Processing: In the Chrysalis framework, processing jobs are performed on the Chrysalis Node itself. This is in the same Docker container that is processing the incoming stream data. Using Chrysalis, stream ingestion and compute happen on the same silicon.
This effectively eliminates latency since “transferring” stream data for computation is as simple as an in-memory copy operation. Also, there is only one failure point. If the stream is up, processing is up. Period.
Finally, let’s discuss data storage / retrieval.
Kinesis Storage/ Retrieval: Kinesis has no built-in mechanism to archive video. If you need to archive video, it needs to be stored separately by a user-created mechanism. There are several ways of going about it but one recommended by AWS is diagrammed to the left. This is complicated and requires additional Amazon services.
Further, Amazon provides no inherent mechanism for retrieving archived video. They do offer the ability to retrieve video that is buffered by a Kinesis stream.
By default, the Kinesis buffer is accessible for 24 hours. You can enable extended data retention for up to 7 days at a minimum additional cost of $14/mo per stream. There is no mechanism to automatically retrieve archived stream data.
Chrysalis Storage/ Retrieval: Chrysalis maintains a short-term stream buffer on each Chrysalis Node. The duration is configurable by the user. If the user chooses, all streamed data over this buffer duration is automatically archived in the S3-compatible cloud storage provider of their choice (for example, Digital Ocean Spaces, Wasabi).
Retrieval requests either return video from the Node’s buffer if the request is recent enough, or automatically from the long-term cloud storage if the request is older than the Node’s buffer. Video can be retrieved either in MP4 segments between specified time ranges, or played using a “YouTube”-style web player starting from any historical time.
Chrysalis Cloud has substantial benefits over using standard AWS services like Kinesis:
- We offer lightweight, user-friendly, inexpensive ingestion that can cut costs by 70%
- Low-latency processing that happens all in one place with fewer failure points
- Easy-to-use buffering and storage that allows for effortless retrieval
- In addition, Chrysalis includes 24/7 built-in monitoring without the need for additional products or services.
We’d love to tell you more and offer you a side-by-side comparison of your current costs vs. what Chrysalis Cloud can offer. Click here to send us a note.