Management roll up of engineering team status and key metrics

This dashboard provides myself and other engineering managers a useful summary of key team performance metrics such as escalations, epics in progress, bug trends and more.

Josip Dlaka

Director of Engineering, SquaredUp

Dashboard Preview

Challenge

This dashboard came about after a chat with Tim, our Director of Engineering Excellence, when he asked, "what would your ideal engineering dashboard look like?".

We agreed that first and foremost, the dashboard needed to give us a clear understanding of how the team is actually performing, without having to dig around in Jira to find out.

As a large engineering team, we use tons of tools day-to-day, and bouncing between disparate data sources for the information we wanted just wasn’t efficient.

We were spending a lot of time trawling through Jira queries and running manual calculations to derive some valuable metrics. And afterwards, spending more time looking for an easy way to share them with the teams that need them.

We needed a simple summary of how the team was getting on, (with access to but not immediate visibility of the finer details) that highlighted anything that needed our attention.

Solution

Using the Jira plugin, we streamed our required metrics into this one dashboard. We now have all of our most important metrics in one place. Everyone in the team, at all levels, has access and visibility.

This dashboard serves two core purposes:

It is perfect for me (Engineering Director) and other managers to dip into whenever we need to view team performance at a glance.
It enables better communication between leadership and the engineers. The dashboard serves as a simple switchboard between myself and the sub teams, who are able to roll up status when something requires attention.

This dashboard benefits everyone within engineering; from management to individual engineers.

It should help create an environment within the team where full visibility allows the appropriate conversations to be had whenever necessary, avoiding a culture of micro-management.

Dashboard walk-through

The first tile you’ll see on the dashboard is the escalation health status block. This shows the current high-level escalation status represented by a Red, Amber, or Green colour.

When clicked on, this drills down to an independent escalation dashboard with greater detail. When it shows green like this, it’s a simple way for me to see at a glance that there’s no escalations that need my attention.

To the right of this are the team health status blocks, which represent each engineering sub-team: Application and Data Visualization.

Like the escalations tile, these status blocks show red, amber or green depending on team performance. When I click into these tiles, I’m taken to each team’s individual dashboard with their own metrics.

They decide what is shown, and simply add monitors to each tile so that when something needs my attention, it is rolled up into this engineering health dashboard for me to see.

Below team health you’ll find two very simple, but effective Gauge tiles showing how many epics each team is currently working on. Each team works on multiple things at a time, so rather than approaching them to understand what they’re keeping busy with, I can just refer to these tiles.

For greater detail, we added these two tables that simply list out the various epics in progress for each team. To learn more, I can just click on these links, and I’ll be taken to Jira to get more detail.

Having these listed and constantly refreshing means I don’t need to spend time digging into Jira to investigate (or waste the teams precious time trying to find out), I can just refer to this dashboard.

We have recently introduced phased deployments, which means that before each update is rolled out externally to all users, it is first deployed to internal tenants. Cancelling or delaying a deployment to external customers is always a last resort, so this gives the team a two-day window to address any issues raised.

The Production Deployment Errors tile shows an “error budget" of five, so every time we need to cancel production deployments due to an escalation or fix, we take one point off. We found the best way to represent this was with the Gauge tile, as it allows us to set milestones of three and four, and enable monitors to alert the team when we hit these. If all five are exhausted each month, the process needs looking at. That’s when we step back, review, and improve going forward.

The large line graph to the right of the dashboard shows the engineering team’s SaaS bug trend. This is a nice trend line that essentially just shows how well we are coping with raised bugs over any given period of time, which you can easily customize. It’s a super useful way to visualize how bug count is growing or declining, and for more detail you can hover over the line to see the exact numbers of bugs and when they were raised.

You can see it peak a few times where we are finding more bugs with each new feature we develop, and then a huge drop in mid-July, thanks to our recent bug bash!You can check out our gamified bug bashboard here.

This table tracks the team’s work in progress limit, calculated as the number of Jira tickets in progress divided by number of team members, + 1. A perfectly balanced team would have a WIP load value of one, so each member is focussed on one task at a time, and anything less than one would indicate that team is underutilized. As you can see from the teams WIP load above, that rarely happens. This tile shows me that the team is spinning too many plates, and it's time for a conversation.

As an example of this working in practice, when I originally saw this number creep up, I spoke with the Data Vis team manager to get to the bottom of it. After chatting it through, we realized that the team were putting Jira automation in a project plan and leaving them there even after the project was deemed complete. This gave the impression that they were doing more than they were.

It was a simple fix, and something we may not have noticed without this tile.

The final tile on the dashboard is the automation tasks open in SaaS. Using a Gauge visualization, this tile shows the number of active Jira tickets related to writing automated tests.

This allows us to track how much effort teams are putting into automating features, tracked through Jira. Going forward, we’d ideally like a little more breakdown to see the type of automation tests, how many we have across each component, and view it in trend lines. This one is a work in progress.

In the future

This is one we’re going to keep building on. I would eventually like to see more tiles related to product quality, such as automation coverage across different parts of the product. Watch this space!