October 4, 2023

The Single Pane of Glass in Modern Observability

Stephen Townshend

Developer Advocate (SRE)

Recently I caught up with Jamie Allen on Episode 67 of the Slight Reliability podcast to discuss the idea of a single pane of glass (SPOG).

Jamie had written an article titled The Single Pain of Glass which coincidentally was what I titled Slight Reliability Episode 10. I thought given our shared use of puns and this topic that it was worth a conversation!

So, what is a single pane of glass? Is it an idea with practical application? How does it fit into the world of modern observability?

What is a Single Pane of Glass?

It was explained to me once using a metaphor. Imagine you are the ruler of a kingdom. A single pane of glass in this context might be a magic mirror which shows you a birds eye view of your kingdom. Just by perusing this magic mirror you can see the wellbeing of your citizens, the progress of your initiatives, and whether there is anything which needs your immediate attention.

Heading back to the real world, a single pane of glass is some kind of central view into the health and status of... something. That something depends on your context:

For an engineer it might be the service they build and operate
For a product owner it might be the products or customer journeys they are working on
For an executive it might be the business divisions they are responsible for

I've heard others criticize the "single pane of glass" concept because it is impossible to show everything you need to know about a system or service in a single view.

If the phrase "single pane of glass" disagrees with you, perhaps you could think of it it as a first pane of glass. A centralized dashboard that tells us it, at a high level, about the overall status of a thing we care, including:

Is there a problem we need to address?
If there is a problem... in which area should we start looking?
...and if we can drill-down further than that to pinpoint what's happening, even better.

At SquaredUp we use the term observability portal as a modern take on the single pane of glass. We think of it as more than a dashboard, but a hub for understanding the state of your systems and services, and an entry point for drilling into the detail.

As Jamie put it in our conversation, you can't track everything at scale. The single or first pane of glass is a tool we can use to keep track of how things are going without overwhelming ourselves and others with detail.

How does it fit into modern observability?

When I think of modern observability I think of detail. Distributed tracing and OpenTelemetry come to mind, as well as vast volumes of telemetry data. So... where does our single pane of glass fit into this?

I want to tackle this in two parts.

SPOG for engineers

As I said earlier, a single pane of glass summarizes the state of something. Provided it distils all the detail into an easy to understand narrative, it can provide a self-service way for other engineers (from other teams) to check the health of your services.

It also acts as a starting point for investigating issues. Modern dashboarding tools provide a way to click on an element to drill-down into the detail. For example, a dashboard for an app indicates the authentication service is having issues. Being able to click on that status and be taken to a more detailed dashboard about that service is helpful.

The opposite of drilling down is rolling up, which is another handy application for the single pane of glass. When teams roll-up the status of the services they own and operate, this can be collated and summarized in a single pane of glass. This gives the teams who own each component ownership of their monitoring, while still keeping leadership informed and aware of the state of things.

You might be thinking... "Why not just use alerts? What value is the dashboard here?". The issue with relying solely on alerts is that they explicitly define thresholds. They don't show more subtle trends or changes over time, such as slowly increasing build times.

SPOG for everyone

From my experience, the single pane of glass concept resonates most with executives. It provides an overview of the state of the business divisions or products they are responsible for. Fundamentally, it helps inform decision making without over-burdening leaders with unnecessary levels of detail.

Here are some other roles who can benefit from the single pane of glass (other than executives):

Product owners, to track whether the features delivered to production are delivering on the hypothesis (based on The Lean Startup approach)
Engineering managers to track the engagement of their teams and the effectiveness of delivery and operations
Customers to check the status of the products or services they are paying for

Caveats and warnings

Dashboards are a bit like documentation. They are an asset that needs to be carefully crafted and maintained. Here's some questions to ask yourself as you build a single pane of glass:

Are you showing vanity metrics? It's tempting to show graphs of data which make us feel good about what we're doing, but that's not the point a single pane of glass. The point is to highlight areas that need attention. A wall of green isn't actually helpful!
Is anyone looking at it? An un-used dashboard is wasteful. Not just the cost of building it, but the effort to maintain it.
How are your graphs being interpreted? If a line graph has a "spike" all of a sudden, it's not unreasonable for someone without context to panic. Think about how you are presenting the data, who is going to be looking at it, and how they might perceive (and react to) it.
Are you trying to include too much? Cramming a dashboard with everything you think you might possibly need is not the way. Focus on key indicators that tell you whether the thing you care about needs attention or not.
Will your SPOG adapt to change? Modern software systems are built on ephemeral and dynamic infrastructure. For example, your service might be hosted on different containers in a different country than they were yesterday. Make sure your single panes of glass are able to accommodate and represent this kind of dynamic situation!

Another challenge that most organizations face is that the data required to build a SPOG is scattered across different tools and locations. How do we pull it all together into a single view?

One approach is to have a layer on top of all of your tools which pulls in the data from each (in realtime). At SquaredUp we're taking this a step further by leveraging a data mesh to correlate objects coming from different tools so that it feels like the data is stored in one location.

Conclusion

A single pane of glass is not about cramming everything we need to know about something into a single view. That's impractical, and frankly impossible. Instead, think of them as:

An overview of the health of something we care about
A starting point for investigation (into issues)
A visual representation of our alerts
A simplified way to share the state of our service to other teams and stakeholders

I would go so far as to say that trying to design a single pane of glass holds value in itself. It forces you to think about what "healthy" looks like for your service or system, and how you can measure that.