Are you really “fine” at 1.2% error rate?

Rethinking video streaming observability with agentic investigation

Did you ever sit in front of a beautiful dashboard, staring at an error rate of 1.2%, and wonder what it actually means for your service?

Is this a good number? Should it be lower? Why is it not at zero? And more importantly, if it is too high, what can you realistically do about it?

The reality of video streaming observability

Observing a video streaming service is inherently complex, and that complexity goes far beyond simply having access to dashboards.

Even with modern observability tooling, you are dealing with multiple dashboards that each show only a partial view of reality. Data comes from across the entire delivery chain, from encoder to packager to CDN and finally the client. Bringing all of these signals together into a coherent understanding still requires a significant amount of manual correlation.

The real challenge, however, goes even deeper.

Streaming errors are not consistent. The same underlying issue can be reported differently depending on the platform, whether it is iOS, Android, a Smart TV, or a browser. On top of that, different players describe issues from their own perspective, often resulting in multiple symptoms for the same root cause. In many cases, these signals are further obscured by cryptic or misleading error codes, as anyone who has worked with AVKit will immediately recognize.

As a result, you constantly find yourself asking whether you are looking at three separate problems or at one issue that simply manifests itself in different ways.

To make sense of all this, you need a deep understanding of video streaming architectures, network behavior and ISP characteristics, as well as the quirks of different devices and client implementations. In practice, this knowledge often resides with a single engineer who is expected to interpret dashboards and decide whether that 1.2% error rate is actually something to worry about.

To avoid spending the entire day staring at dashboards, engineers typically rely on alerts to highlight potential issues. However, these alerts are often based on arbitrary thresholds that fail to capture the nuances of real-world streaming behavior. As a result, long tail delivery issues remain invisible, while both dashboard fatigue and alert fatigue continue to grow. In this situation, the last thing anyone needs is yet another tool that presents the same data from a slightly different angle.

There must be a better way

While working with our customers on detecting and debugging streaming incidents, we kept coming back to a simple but fundamental question.

Why do humans have to actively search for issues in the first place? Why should it not be the other way around, where the issue finds you?

Meet FOCUS: an agentic approach to observability

This thinking led us to build something fundamentally different.

Instead of following the traditional flow from dashboards to human interpretation to insights, we built:

FOCUS — an ML and AI based video streaming observability agentic framework

We believe you should not have to find the problems. The problems should find you. FOCUS enables exactly this.

Data is the foundation, but not the solution

Capturing detailed streaming telemetry is the foundation of any observability system.

Standards such as CMCD v2 play an important role in bringing more structure and consistency to this data, enabling a more unified view across the ecosystem.

However, even with perfectly structured and complete data, the core problem remains.

Data alone does not solve the problem.

Finding the invisible through ML driven clustering

To move beyond dashboards, we built a set of machine learning algorithms that identify clusters of interest across the available streaming sessions.

Instead of focusing on predefined thresholds, these algorithms look at the data from multiple perspectives and uncover patterns that would otherwise remain hidden. This can include degraded ISP performance in a specific region, a failing media item buried deep in the catalog with only a handful of daily views, DRM subtitle issues affecting a very specific device configuration, or delivery inconsistencies in a multi CDN setup.

These are real world issues that impact actual viewers, yet they remain hidden behind an aggregated error rate such as 1.2%. FOCUS discovers unusual playback patterns, investigates their significance and delivers relevant findings to the right team.

Grounded in domain knowledge

Detecting patterns is only useful if they can be interpreted correctly. Every investigation in FOCUS is enriched with two layers of knowledge.

The first is a general understanding of video streaming, covering how protocols behave, which codecs are supported on which platforms, and what typical failure patterns look like. The second, and arguably more important layer, is deeply integrated customer specific context.

The same principle applies across the board, whether it involves known client quirks on a specific device, framework behaviors that produce misleading error codes, or platform specific edge cases in a multi CDN setup. By embedding this operational knowledge directly into the system, we give each investigation the context needed to test its explanations.

In essence, we codified the knowledge that a senior video streaming engineer would need in order to correctly interpret all of this data.

From signals to real issues through agentic investigation

An unusual pattern is a signal for investigation. FOCUS checks playback impact and service context before deciding whether it warrants an incident. Retries, expected platform behaviour and genuine failures can produce superficially similar metrics.

This is where the agentic investigation framework becomes critical.

Agents examine affected sessions, compare relevant delivery evidence and use streaming-specific checks. Domain knowledge and customer context help them test explanations, assess relevance and document what remains uncertain.

From detection to action

Once a signal has been validated as a relevant issue, the system moves beyond detection.

It automatically creates an investigated ticket that includes a clear description of the problem, its impact on users, the likely cause and any remaining uncertainty. This information is then delivered directly into the tools that teams already use, whether that is Jira, Slack, Teams, email, or other systems.

There is no need to switch environments, as the system integrates directly into existing workflows.

The impact

The aim is to reduce manual discovery, correlation and triage. Generated findings bring the observed problem, its impact and the supporting evidence into the team's existing workflow.

Examples of the problems session context can reveal

Delivery changes associated with abnormal session endings.
Protected subtitle handling affecting particular device cohorts.
Content unavailable through one route in a multi-CDN setup.
Repeated delivery errors associated with playback disruption.

The investigation connects an observed pattern to the available technical evidence. Coverage and diagnostic detail depend on the data collected for the case.

Why not just use AI?

A common question is why not simply apply AI to the data and let it figure things out. The reality is that without the right structure, context, and domain specific knowledge, this does not work in practice.

👉 Ralf explains this in more detail here.

See it in action

Explore FOCUS or book a demonstration to see the workflow. Explore FOCUS · Book a demo

Why this matters

Useful observability connects a symptom to evidence that a team can act on. This is the work FOCUS is designed to automate.

Final note

A huge thank you to the whole team, and especially to Andreas Sapountzis and László Schmidt, for building this continuously evolving system.

By moving away from dashboards and toward automated, actionable insights, we enable engineers to focus on what really matters, which is improving the quality and reliability of their streaming services.

Curious to learn more? Let’s talk.