Video Encoding with Bitmovin and Scality

by Laure Vergeron | Mar 21, 2017 | Data management, Tutorials

Bitmovin provides a dedicated service, enabling live and on demand encoding of videos into adaptive bitrate formats such as MPEG-DASH and HLS in the cloud. This service comes with a comprehensive API that allows seamless integration into any video workflow.

Recently Bitmovin released a managed on-premises encoding solution on top of Kubernetes and Docker that works for VoD and Live and offers the same features as their cloud encoding service. Managed on-premises encoding offers the benefits of Software as a Service solution while utilizing your own infrastructure.

With the release of their managed on-premises encoding, they also released support for the Scality RING storage, allowing this storage solution to be used in the private cloud. In the following tutorial we will describe how to setup a Scality S3 Server [1] and use it together with the Bitmovin API to download input assets from, and upload the encoded content to.

Setup a Scality S3 Server Storage

In this tutorial we are using the official Scality S3 Server Docker image [2] that allows for a very easy and fast setup. You will need to have Docker installed in order to follow the below steps. We will be using a persistent storage, in order to keep the files we copy to the Scality S3 Server.

With the following command you can start the Scality s3 Server:

sudo docker run -d --name s3server -p 80:8000 \
  -e ACCESS_KEY=accessKey1 \
  -e SECRET_KEY=verySecretKey1 \
  -e HOST_NAME=scality.bitmovin.com \
  -v /mnt/s3data:/usr/src/app/localData \
  -v /mnt/s3metadata:/usr/src/app/localMetadata

This will launch the Scality S3 Server and bind the service to port 80 on your instance. The two environment variables ACCESS_KEY and SECRET_KEY allow you to setup authentication credentials to access the Scality S3 Server.

If you plan to access the service using a DNS name (e.g., scality.bitmovin.com), you must set the environment variable HOST_NAME accordingly. The service will deny requests if the request headers do not match with this value. Alternatively, you can also use an IP without the need to set any hostname. The volume mounts are used to persistently save the data of the Scality S3 Server to directories in your filesystem.

For more information and configuration options refer to the Docker manual [3] at the Scality GitHub project.

With that we have a running Scality S3 Server up and running. To test it with the Bitmovin API we need to create a bucket and upload a test asset into the bucket. For doing that we can use the generic s3cmd command line tool from AWS.

To access your Scality S3 Server with s3cmd a configuration file is required. The following will show an example of a configuration file that will allow access to the just created Scality S3 Server:

[default]
access_key = accessKey1
secret_key = verySecretKey1
host_base = scality.bitmovin.com:80
host_bucket = %(bucket).scality.bitmovin.com:80
signature_v2 = False
use_https = False

Save the configuration file e.g., as scality.cfg so you can directly use it in your s3cmd command or at ~/.s3cfg in which case you do not need to explicitly specify a configuration file.

Create a bucket with s3cmd:

s3cmd -c scality.cfg mb s3://testbucket

Verify the bucket got created:

s3cmd -c scality.cfg ls

Upload a test asset to the bucket:

s3cmd -c scality.cfg put samplevideo.mp4 s3://testbucket/inputs/

Check if the test asset got correctly uploaded:

s3cmd -c scality.cfg ls s3://testbucket/inputs/

Using Scality with the Bitmovin API

Bitmovin added support for the Scality S3 Server with a generic S3 interface. In the following we will discuss an easy example on how to use a Scality S3 Server for retrieving an input asset, as well as for storing the encoded output back to the Scality S3 server. For the sake of simplicity we will be using the same Scality S3 Server for input and output that we have just created above.

Obviously, you could also use different Scality S3 Servers for input and output.

For this tutorial we will be using Bitmovins PHP API client that already has a neat example [4] of how to use a Scality S3 Server for retrieving an input asset and uploading the encoded data back to the Scality S3 Server.

To get the Bitmovin PHP API Client you can either download it from GitHub [5] or install it using composer. Please see the API clients repository for more information about the setup.

First of all we need to specify all data that is required to run the example. In the following we will be using the data from the above Scality S3 Server that we have just created referencing the uploaded input file samplevideo.mp4. We are also specifying an output folder where the encoded files should be placed:

$scalityHost = ‘scality.bitmovin.com’;
$scalityPort = 80;
$scalityAccessKey = 'accessKey1';
$scalitySecretKey = 'verySecretKey1';
$scalityBucketName = 'testbucket';
$scalityInputPath = "inputs/samplevideo.mp4";
$scalityOutputPrefix = "output/samplevideo/";

Initialize the Bitmovin API Client

$client = new BitmovinClient('INSERT YOUR API KEY HERE');

For initializing the BitmovinClient you need to have an account with Bitmovin and the API key of your account available.

Create an input configuration

We will create an input referencing the samplevideo.mp4 from our Scality S3 Server.

$input = new GenericS3Input($scalityBucketName, $scalityAccessKey, $scalitySecretKey, $scalityHost, $scalityPort, $scalityInputPath);

Create an output configuration

We will create an output configuration that will allow us to store the encoded files to our Scality S3 Server in the output/samplevideo folder.

$output = new GenericS3Output($scalityAccessKey, $scalitySecretKey, $scalityHost, $scalityPort, $scalityBucketName, $scalityOutputPrefix);

Create an encoding profile configuration

An encoding profile configuration contains all the encoding related configurations for video/audio renditions, as well as the encoding environment itself. Choose the region and cloud provider where the encoding should take place. Of course it is optimal if it is close to where your Scality S3 Server is located to reduce the download and upload times 😉 If you are using Bitmovins on-premises feature, you can simply choose your connected Kubernetes cluster instead of a cloud region and the encoding will be scheduled on your own hardware.

$encodingProfile = new EncodingProfileConfig();
$encodingProfile->name = 'Scality Example';
$encodingProfile->cloudRegion = CloudRegion::GOOGLE_EUROPE_WEST_1;

Add video stream configurations to the encoding profile

In the following you will see a configuration for a 1080p H.264 video representation. You will want to add more video representations for your ABR streams as also shown in the example in our GitHub repository.

$videoStreamConfig_1080 = new H264VideoStreamConfig();
$videoStreamConfig_1080->input = $input;
$videoStreamConfig_1080->width = 1920;
$videoStreamConfig_1080->height = 1080;
$videoStreamConfig_1080->bitrate = 4800000;
$encodingProfile->videoStreamConfigs[] = $videoStreamConfig_1080;

Add an audio stream configuration to the encoding profile

$audioConfig = new AudioStreamConfig();
$audioConfig->input = $input;
$audioConfig->bitrate = 128000;
$audioConfig->name = 'English';
$audioConfig->lang = 'en';
$audioConfig->position = 1;
$encodingProfile->audioStreamConfigs[] = $audioConfig;

Create encoding job and start it

The JobConfig acts as a container for all the previously created configurations. Here we also define that we want to have MPEG-DASH and HLS output created. The JobConfig object will be passed to the BitmovinClient, that will then start the encoding job and wait until it is finished.

$jobConfig = new JobConfig();
$jobConfig->output = $output;
$jobConfig->encodingProfile = $encodingProfile;
$jobConfig->outputFormat[] = new DashOutputFormat();
$jobConfig->outputFormat[] = new HlsOutputFormat();
$client->runJobAndWaitForCompletion($jobConfig);

After the encoding job has finished you should have all encoded files for MPEG-DASH and HLS with the manifests on your Scality S3 Server. When using Scality you can simply access the files with HTTP. For the above example the HTTP links would be as follows:

MPEG-DASH: http://scality.bitmovin.com/output/samplevideo/stream.mpd

HLS: http://scality.bitmovin.com/output/samplevideo/stream.m3u8

[1] The Scality S3 Server is an open-source object storage project to enable on-premise S3-based application development and data deployment choice

[2] https://hub.docker.com/r/scality/s3server/

[3] https://github.com/scality/S3/blob/master/DOCKER.md

[4] https://github.com/bitmovin/bitmovin-php/blob/master/examples/CreateSimpleEncodingWithScalityInputAndOutput.php

[5] https://github.com/bitmovin/bitmovin-php

Thanks to the Bitmovin R&D Team for their contribution!