Building effective KPI dashboards that don't average away the truth

KPIs can be used in SquaredUp at different levels, from an individual service metric to a view of KPI health across an entire organisation.

This post walks through a set of dashboards built around availability, performance, cost, and incidents, and how I’ve used KPI monitoring and roll-ups to bring those views together.

We’ll start with individual KPIs, then look at how related KPIs can be rolled up and surfaced in a higher-level dashboard, with the underlying detail still available to explore.

Building an enterprise KPI dashboard

This is the Enterprise KPI Dashboard. I built it around the areas a team would typically want to keep an eye on: availability, performance, cost, and incidents. The tiles use data from sources including AWS, SCOM, VMware, Dynatrace, and ServiceNow.

John Hayes’ post on KPI roll-ups introduced the idea of tracking the same KPI across different teams and regions, using sales enquiry response time as an example. I’ve applied a similar approach here to service response times, bringing related KPIs together into a roll-up while keeping the individual KPIs underneath.

In this dashboard, I have separate response-time KPIs for different services. Each KPI continues to show the performance of its individual service, while the roll-up gives me an overall view of their state.

For this roll-up, I used Top as the aggregation. This means the highest response-time value determines the state of the roll-up.

With a Critical threshold of greater than 1,000 ms, the roll-up becomes Critical id any service crosses the threshold. The overall state is visible at a glance, while the individual KPIs remain available underneath. I can drill down to see which service is responsible for the Critical state rather than losing that detail in the summary.

This connects to something I explored in my earlier post, How KPIs lose their meaning and what to do about it. When several signals are reduced to a single KPI or status, the rule used to create that summary matters.

Here, that rule is explicit: the highest response time determines the overall state.

KPI summary for your entire organization

The same KPI rollup capability can also be used to create a higher-level view across your organization.

For this dashboard, rather than showing every individual KPI, I wanted to give a quick summary of how many KPIs are being tracked and how many are currently at risk.

The dashboard groups the KPIs by category, such as Availability, Tickets, Rate, Error, Duration, and Cloud Cost. Each category shows the total number of KPIs and, where applicable, how many are currently at risk.

For example, you might see:

• 18 Availability KPIs, with 1 at risk
• 8 Ticket KPIs, with 0 at risk
• 5 Duration KPIs, with 3 at risk
• 4 Cloud Cost KPIs, with 2 at risk

This gives you a quick view of where attention may be needed without filling the dashboard with every individual KPI.

The individual KPIs aren't lost, though. You can drill into a category to see the KPIs behind the number, and from there understand exactly which KPI is contributing to the rollup and why it is at risk.

This makes the rollup useful at two levels: the summary gives you the quick organizational view, while the drill-down takes you back to the individual KPIs when you need more detail.

Seeing KPI health at a glance

KPIs give you a consistent way to track the things that matter across your environment. Once you have them in place, KPI rollups give you another way to use that information — whether that's bringing together related KPIs for a service or giving the wider organization a quick view of KPI health.
The dashboards above are two examples of how I've used them. One gives you the detail; the other gives you the bigger picture, with the option to drill down when you need to see what's behind the numbers.

Want to try building something similar with your own data?