How we monitor Azure DevOps internally at SquaredUp

Dashboard Preview

Our engineering team uses SquaredUp dashboards internally to monitor our Azure DevOps pipelines. In this article, our Director of Engineering Excellence, Tim Wheeler, walks us through the dashboards we use for each of our different services.

Dashboard walk-through

SquaredUp Platform dashboard for DevOps

This is the Azure DevOps dashboard we use internally for a full overview of ADO that lets us react to immediate issues and track our efficiency long-term. The most important metric, in the top left, is the scalar metric showing ‘Build Failures in Master’. Directly beneath, you can see the ID of the pipeline that has failed, in red.

We also get visibility over anything that is stuck in a long build duration and have built an alert for that. Plus, we can see the most common build stage failures to keep track of where we need to improve.

In the second row of visualizations we can also see job queues and agent usage to track efficiency.

Scrolling down the dashboard, we have build durations over the past 7 days and the build run details.

Even further down the dashboard is a table that shows the build runs. The red dot immediately identifies the failed build and even which stage has failed . Then use the hyperlink to drill directly into Azure DevOps platform to see the error in detail.

This table uses a data stream to return all the stages within a pipeline.

If you use work items, you can use the work item query language to write a data stream that will pull back any work item that you use, whether for a use case, test case, or theme. There are plenty of options open to you.

Builds Focused dashboard

There’s also a Builds Focused Azure DevOps dashboard that we use to track all the most important metrics directly related to builds.

The ‘Main branch Build Failures’ tile is a scalar that counts the number of build pipelines that have failed in the last 24 hours. We are also monitoring this tile so that if this number is above ‘0’ then the tile and dashboard will go red. This means you should scope this tile to your most critical pipeline.

The ‘Build Runs’ tile is a health blocks visualization where each tile is showing the status of a pipeline result.

‘Build Duration for Main Branch’ is displaying a line graph of the duration of a specific pipeline.

'Task Failures’ is a Donut visualisation that lets you easily see which tasks are the most common cause of a failure in your pipelines.

These two tiles allow you to look at the performance of your pipelines by looking at the size of job queues and how many agents are in use at a specific time.

DevOps overview dashboard

Our DevOps straightforward overview dashboard shows which dashboard we need to worry about and helps us keep track of our overall Azure costs.

Dashboard Server product overview

Finally, our Azure DevOps dashboard looks very different for our on-premises product because failures aren’t a reflection on what we’re putting into production. So, we care more about job queues and build durations because they’re impacting our engineers. You have the flexibility to tailor your dashboards to show what’s most important to your team.

Get your dashboards

All of these Azure DevOps dashboards have been built using our out-of-the-box dashboards as a starting point, so if you'd like to recreate them for your organization, you can expect to be up and running in minutes.

1. Create a free SquaredUp account
2. Add the Azure DevOps data source
3. Just hit the toggle to automatically add out-of-the-box dashboards to your workspace, and instantly view your live data

To see what other dashboards you can create, check out our Dashboard Gallery.