Introduction
Running a streaming service at large scale is a huge technical challenge. Major problems, especially during peak events, can be caused by unpredictable peering limitations between Content Delivery Networks (CDNs) and the variety of Internet Service Providers (ISPs) used by the audience across different regions.
If the peering is limited within a specific area or any involved infrastructure capacity suddenly decreases, the user experience can be impacted heavily by poor video quality or frustrating rebufferings.
Approach and goals
To address this challenge, EINBLIQ.IO teamed up with international partners. The goal was to set up a quality-aware multi-CDN system which is driven by real-time streaming and network performance metrics. In this Proof of Concept (POC) the industry standards Common Media Client Data (CMCD) and Content Steering are used to achieve two key goals:
In this blogpost you will learn…
Common Media Client Data (CMCD)
One of the technologies used in this POC is CMCD. CMCD is an open specification, published by the Web Application Video Ecosystem (WAVE) project within the Consumer Technology Association (CTA), with the goal to standardize video playback and quality metrics.
CMCD describes a set of data points such as:
- Video bitrate
- Network throughput
- Streaming format
- …
These data points are captured by the video player on the client’s device first, and then sent to the CDN. From there, they can be collected and processed to understand the streaming performance of each single video stream.
CMCD is widely adopted in the market, as most of the major video players now support CMCD, such as Android’s ExoPlayer, dash.js, hls.js and RDK’s AAMP player implementation, which is the native video player for numerous big screen products.
As CMCD is already implemented in most of the major player frameworks, no additional adaptations or player side analytics integrations are required to be implemented by the content provider in order to collect these basic playback metrics and to understand the streaming performance.
A further advantage: As CMCD describes a common set of playback metrics, these metrics are comparable between different players and application environments.
Content Steering
Besides CMCD, this POC relies on the technology of Content Steering to easily set up a multi-CDN environment.
Content Steering enables a dynamic mechanism for distributing and moving video streaming sessions across different CDNs.
To do so, the video player regularly contacts a Content Steering server which in turn responds from which CDN the player should fetch the video content.
Content Steering was introduced by Apple in 2021 and is standardized for HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (MPEG-DASH), which are the dominant streaming formats used today.
Furthermore, a growing number of video players support Content Steering out of the box, among them the native Apple AVPlayer, which makes Content Steering immediately available for all iPhones, iPads and Apple TVs. In addition, hls.js and dash.js already support Content Steering, which makes it widely available in web environments, as many popular players use these frameworks internally. Starting next year, also ExoPlayer – the core video player for Android – is expected to join the list.
With Content Steering, content providers have unprecedented control over the delivery of their content. It is now possible to switch even single users mid-stream from one CDN to another. And all this happens invisibly to the user.
If you want to learn more about Content Steering, you can read more about it in our blog articles about the Basics of Content Steering and how Content Steering enables quality aware Multi-CDN. Also, don’t forget to check out our live browser demo.
Goal #1: Understanding regional streaming and network performance in real-time
Technical Setup
Now, as we have an overview of the technologies which are used in this POC, we are looking more closely at the infrastructure that was set up to realize the described scenario.
Figure 1: General POC setup to capture CMCD information from two CDNs
The video content, in this case an MPEG DASH livestream is provisioned through one of our partners’ origin.
The livestream is picked up by two CNDs (“cdn1” and “cdn2”), both distributing the MPEG DASH livestream. Multiple CDN vendors are supported and have been tested, including (but not limited to) Akamai, AWS CloudFront, Fastly and Azure Front Door. Different metrics capturing approaches are supported by EINBLIQ.IO and data formats are ultimately harmonized in a common backend.
For the player side, a CMCD-enabled RDK-based AAMP player is used. Its instances stream the content from both CDNs in a pre-defined split. The player uses an out-of-the-box RDK AAMP version, which did not need any adaptations.
In addition to the RDK player, EINBLIQ.IO prepared a dash.js-based web application to showcase the POC with multiple player technologies. The player is based on the latest unmodified dash.js version, which already supports CMCD and Content Steering by default.
No further adaptations are needed. The web-based application also includes a visual status feedback for the demo, indicating the current video quality, the connection speed and the current CDN in use by this player instance:
Figure 2: Test dash.js-based video player used in POC (image blurred)
While streaming the video content from the two CDNs, both players, the RDK-based AAMP player and the dash.js-based web player, are transmitting CMCD information back to the CDN.
The players are continuously requesting new video segments from the CDN, with each request containing updated CMCD information.
In case of the AAMP player, the CMCD information is sent in predefined HTTP request header fields, while the dash.js player is sending the CMCD information encoded within the query string.
Both options – CMCD in HTTP headers and CMCD in the query string – are defined in the CMCD specification and are supported by EINBLIQ.IO.
Now, as the CMCD information arrives at a given CDN, it needs to be forwarded to the EINBLIQ.IO API endpoint, where it is processed and harmonized to a common data format before being analyzed in near real-time.
To achieve this, the different CDNs setups were extended by EINBLIQ.IO to enable metrics forwarding to the central API endpoint. The implementations leverage the different CDNs’ data forwarding features to keep adaption efforts minimal.
The last element of this part of the chain is the EINBLIQ.IO CMCD insights API endpoint. This endpoint captures and harmonizes CMCD information provided by the different CDNs. The EINBLIQ.IO cloud backend then processes this information in near real-time to visualize it and to provide aggregated actionable insights via an API. A main use case is the provision of all information required to drive a Content Steering multi-CDN decision engine based on live quality data – more under Goal #2.
POC Demo
To demonstrate Goal #1 – understanding the regional streaming and network performance in real time – a test campaign was executed with clients across different countries, regions, and ISPs.
In total:
- around 5.8 million CMCD sets
- across 3 countries
- across 18 regions
- and 15 ISPs
were processed during the execution of the test.
Figure 3: High-level CMCD statistics from POC
Figure 4: Map with U.S. locations of test streams for POC
To understand the regional streaming and network performance, all incoming CMCD information was harmonized and aggregated to derive regional streaming quality metrics.
The outcome was a live dataset with streaming quality data for CDN, region and CDN. As an example, Figure 1 showcases streams delivered within the United States using different CDNs across different ISPs in different regions:
Figure 5: Streaming quality measurement during POC
While in an ideal scenario, all streams would be provided at optimal quality everywhere, anytime and independent of the CDN and ISP in use, this is often not the case. Not all combinations of CDNs and ISPs provided the same level of performance with again differing results in different regions.
This already gives a glimpse into how different the experience can actually be for the individual consumer.
Moreover, this performance can change quite significantly over time.
As an example, this is the change in streaming performance over the course of 6 hours for the same ISP in one region using two different CDNs:
Figure 6: Fluctuating streaming performance over 6 hours (same ISP, same region, 2 different CDNs)
Goal #2: Switching selected users mid-stream to the best available infrastructure
Technical Setup
After reaching Goal #1, we are now able to understand regional streaming performance in near real-time.To reach Goal #2, we used these insights to switch users mid-stream to their individually best CDN.To do so, the infrastructure set up for goal #1 was extended by the EINBLIQ.IO Content Steering Service:
Figure 7: POC setup for CMCD collection and Content Steering
The EINBLIQ.IO Content Steering Service is capable of switching users between different CDNs seamlessly.
Each video streaming client supporting Content Steering periodically requests a so called “Steering Manifest” from the Content Steering Service. This is a simple JSON payload, which describes the current prioritization of all available CDNs individually for each client. The client then picks the CDN with the highest priority.
If the client is being redirected to another CDN, the switch happens mid-stream and without any noticeable interruption to the user.
To reach POC Goal #2, the dash.js-based web player from EINBLIQ.IO was used, as the RDK-based AAMP player is not yet fully ready to support Content Steering.
To understand which is the best available CDN for each client, the Content Steering Service fetches the regional streaming performance information in real-time from Goal #1.
By exploiting these insights, the Content Steering Service is not only able to understand the current streaming and network performance of each client which requested the “Steering Manifest”, but also to identify if there is a better CDN available for a Client in the specific region connected to a specific ISP.
POC Demo
The demo was run with multiple Content-Steering-enabled clients across different regions and Internet Service Providers.
For these streams, an overview visualization shows the current streaming performance per CDN, ISP and region. In a simplified view, the optimal CDN was listed (Figure 8) to be prioritized by the individual client cohorts.
Figure 8: Dashboard table showing the best CDN for a given region and ISP combination from the POC
To demonstrate also on the client side which CDN is used at any given time during the POC, the EINBLIQ.IO dash.js-based web application shows the current CDN within a status bar. This makes any switch visible immediately to the test user while the switch is imperceptible in the video itself.
Once the clients are up and running, we can now start to switch specific clients between the available CDNs to improve their users’ streaming experience.
As an example, a test client in the region of Bavaria connected to a local ISP was served with a livestream from a given cdn1 (Figure 9). Likely due to a non-ideal connection quality between the ISP and cdn1, the playback only happened in a lower resolution.
After manually switching this client to cdn2 using the Content Steering Service, the playback performance could be increased as the video bitrate picked by the player went up to full 8Mbit/s shortly after. These insights were derived from the CMCD sets collected on both CDNs.
Based on these insights, it became obvious that the peering quality between this ISP and cdn2 clearly out-performed that with cdn1 during the test.
Figure 9: Performance comparisson for different CDNs in the same region using the same ISP
Figure 10: An example showing how the technology of Content Steering together with real-time CMCD information can be used to optimize the playback performance for individual sessions or groups of sessions (image blurred)
Outcome and next steps
In this POC, together with its partners, EINBLIQ.IO showcased successfully that combining the two technologies CMCD and Content Steering can overcome everlasting challenges in video streaming. Ultimately, the tested approach helps to deliver a better and more reliable streaming experience.
To address (sudden) peering bottlenecks between specific CDNs, regions and ISPs, it is instrumental to:
- Understand regional streaming and network performance issues in real-time
- Derive steering decisions on these metrics for distributing clients to the best available CDN based on their context
All this helps to provide a stellar video streaming experience to all users – even when peak usage or high traffic amounts cause local bottlenecks.
As this POC was executed within a confined testing environment, the POC partners now plan to extend the trials to more and real users.
Did we spark your interest?
Are you interested in further practical applications of CMCD and Content Steering? Do you consider participating in the evolution of this or a similar POC? Are you curious how EINBLIQ.IO can help you to improve the video streaming quality for your service while reducing the workload for your skilled staff?
We invite to reach out and to join the conversation: https://einbliq.io/contact/