10 metrics every Azure DevOps engineer should monitor

As a DevOps engineer maintaining deployment pipelines in Azure DevOps, it's crucial to monitor several aspects to ensure the pipelines are efficient, reliable, and secure.

Why monitoring Azure DevOps metrics matters

Whether you’re deploying your first pipeline or scaling across teams, understanding your Azure DevOps data is the key to smoother releases and fewer surprises. By focusing on the right metrics, you empower your team to spot issues early, improve quality, and deliver value faster.

Let’s break down the 10 metrics that matter most – what they mean, why they’re important, and how you can act on them.

1. Pipeline execution and success rates

The percentage of builds and deployments that succeed versus those that fail.

Why it matters: Frequent failures slow down your team and can hide deeper issues. Tracking this helps you catch problems early and keep your pipeline healthy.

How to use it: Set a baseline and investigate any dips. Look for patterns—are certain branches or environments less reliable?

The metrics to track

• Build success/failure rates: Track how often builds succeed or fail.
• Deployment success/failure rates: Monitor the success rates of deployments across different environments (e.g., dev, staging, production).
• Deployment frequency: Track how often deployments happen in different environments to assess deployment cadence.

How to use it with SquaredUp:

• Use SquaredUp to display pipeline execution counts, success rates, and failure patterns at a glance.
• Drill down into failed runs to quickly identify root causes and resolve issues.
• Share real-time status with your team, so everyone stays informed and can act with confidence.

2. Resource or pipeline and agent utilization

This metric shows how efficiently your build agents and pipelines are being used – tracking queue times, job durations, and idle resources.

Why it matters: Underused resources waste money; overused ones slow down builds and deployments.

How to use it: Monitor queue times and agent usage. If jobs are waiting, consider adding more agents or optimizing your pipeline steps.

The metrics to track

• Agent utilization: Monitor the usage of build agents, ensuring they are efficiently allocated.
• Pipeline queue times: Keep an eye on how long jobs wait in the queue before starting, as long queue times can indicate resource constraints.

How to use it with SquaredUp:

• Visualize agent usage, pipeline queue times, and job durations in real time with SquaredUp dashboards.
• Identify bottlenecks or underused resources so you can balance workloads and scale efficiently.
• Monitor trends to plan for future capacity and avoid unnecessary spending.

3. Code quality metrics

Code quality metrics track the health of your codebase, including test coverage, code complexity, and the number of bugs or vulnerabilities detected1.

Why it matters:
High-quality code is easier to maintain, less prone to bugs, and safer to deploy. These metrics help you spot problem areas and improve your development process.

The metrics to track

• Test coverage: Ensure that unit tests cover a significant portion of your codebase.
• Code analysis results: Use tools like SonarQube or Azure DevOps’s built-in tools to monitor code quality, looking for issues like technical debt, code smells, and security vulnerabilities.
• Test failure rates: Track the frequency of test failures to catch recurring issues.

How to use it with SquaredUp:

• Integrate code quality tools with SquaredUp to visualize test coverage, code smells, and vulnerability trends.
• Build dashboards that highlight areas needing attention, so your team can prioritize improvements.
• Use SquaredUp’s clear, customizable views to share code quality insights with both technical and non-technical stakeholders.

4. Security and compliance

Monitoring for vulnerabilities in dependencies and ensuring secrets are managed securely.

Why it matters: Security isn’t optional. Automated checks help you catch issues before they reach production.

How to use it: Set up automated dependency scanning and secret management. Review audit logs regularly for unauthorized changes.

The metrics to track:

• Dependency scanning: Monitors for known vulnerabilities in third-party components and ensures compliance with licensing and policy requirements.
• Pipeline security: Ensure that sensitive data (e.g., secrets, credentials) is securely managed within the pipeline.

How to use it with SquaredUp:

• Surface security alerts, permission changes, and audit logs on custom dashboards for full visibility.
• Monitor compliance checks and policy adherence across all projects and environments.
• Get notified of critical security events, so you can act fast and minimize risk.

5. Performance and availability

These metrics measure how well your applications and infrastructure are performing, and how often they’re available to users.

Why it matters:
Consistent performance and high availability keep users happy and reduce business risk. Proactive monitoring helps you fix issues before they impact customers.

The metrics to track:

• Average build/deployment time: Measure how long builds and deployments take to identify bottlenecks.
• Response times: Measure the time it takes for the pipeline to respond to triggers, which can impact overall deployment speed.
• Pipeline downtime: Track any periods where the pipeline or parts of the Azure DevOps service are unavailable.

How to use it with SquaredUp:

• Connect SquaredUp to your performance and uptime data sources for a unified, real-time view.
• Set up dashboards to track response times, error rates, and availability—all in one place.
• Use SquaredUp’s drill-down features to investigate performance dips or outages and quickly identify root causes.

6. Artifact management

Artifact management tracks the storage, retention, and access of build outputs, packages, and other deployment files.

Why it matters: Unmanaged artifacts can quickly eat up storage and costs.

How to use it: Set retention policies and regularly review what’s stored. Clean up old or unused artifacts.

The metrics to track:

• Artifact storage usage: Monitor the storage usage for build artifacts to ensure you aren’t running out of space.
• Versioning and retention: Keep an eye on how many versions of artifacts are stored and manage retention policies.

How to use it with SquaredUp:

• Use SquaredUp to monitor artifact storage, retention policies, and usage trends.
• Visualize which artifacts are consuming the most space and identify those that can be cleaned up.
• Track downloads and access to ensure only authorized users are retrieving sensitive builds.

7. Environmental Metrics

Environmental metrics monitor the health, deployment history, and resource usage of your DevOps environments—like Development, Test, Staging, and Production.

Why it matters:
Healthy environments mean smoother deployments and fewer production surprises. Tracking these metrics helps you understand the impact of each release.

The metrics to track

• Environment availability: Ensure that the environments where code is deployed are available and ready for deployments.
• Deployment success: Check the last deployment to that environment was successful.

How to use it with SquaredUp:

• Use SquaredUp to visualize the status and health of each environment at a glance.
• Track deployment frequency and success rates for every environment, helping you spot issues before they reach production.
• Build custom dashboards that show resource usage and deployment trends, making it easy to optimize and troubleshoot.

8. User Activity and Permissions

This metric tracks who’s accessing your DevOps environment, what they’re doing, and what permissions they have.

Why it matters:
Monitoring user activity and permissions is essential for security and compliance. It helps you spot unusual behavior and ensure only the right people have access.

The metrics to track:

• Audit logs: Regularly review audit logs for any unauthorized access or changes to the pipeline configuration.
• Permission changes: Monitor who has access to what in your Azure DevOps project to prevent unauthorized changes.

How to use it with SquaredUp:

• Build dashboards that surface user activity, highlighting key actions and changes in permissions.
• Visualize access patterns and quickly audit permissions across teams and projects.
• Set up notifications for permission changes or suspicious activity, so you can respond quickly and maintain compliance.

9. Feedback Loops

What it is:
Feedback loops are the signals your team receives – automatically or manually –about the health of your code, pipelines, and releases. This includes build results, test outcomes, and user feedback.

Why it matters:
Quick, clear feedback helps you catch issues early, improve quality, and keep projects on track. Tight feedback loops mean faster learning and fewer surprises.

The metrics to track:

• User feedback: Implement mechanisms to gather feedback from developers and stakeholders on the pipeline’s performance and usability.
• Automated feedback: Use automated tools to report on post-deployment issues like performance degradation or increased error rates in the target application.

How to use it with SquaredUp:

• Build dashboards that visualize build and test results in real time, so your team sees issues as they happen.
• Surface trends in feedback cycles to spot bottlenecks or recurring problems.
• Use SquaredUp’s integrations to bring together feedback from multiple sources—code, pipelines, and even user surveys—into one clear view.

10. Cost Management

Cost management tracks your cloud spending across services, resources, and environments.

Why it matters: Keeping an eye on costs helps you stay within budget and justify investments.

The metric to track:

• Resource costs: Monitor the costs associated with running pipelines, including build agents and artifact storage.

How to use it with SquaredUp:

• Use SquaredUp’s Azure Cost Management dashboards to visualize spending by service, resource group, or location in real time.
• Filter and group costs however you need—by subscription, app, or department—to spot trends and anomalies fast.
• Set up alerts to notify you of unexpected spikes, so you can take action before costs get out of hand.

Keep Exploring

• 3 Powerful Tools for Reporting Azure DevOps Metrics
• How We Monitor Azure DevOps Internally at SquaredUp
• Why Native Azure DevOps Dashboards Fall Short

How to monitor these metrics

These monitoring practices will help ensure that your pipelines are running smoothly, with minimal disruptions and maximum efficiency. But how and where do you monitor these?

The built-in options for monitoring Azure DevOps are fairly limited. If you're familiar with Azure Monitor then particular Application Insights is a good option to go deep on troubleshooting – here's a blog that covers this topic in detail.

If you're looking for a simpler option, SquaredUp is the easiest way to get up and running with dashboards and monitoring for Azure DevOps. It comes with a selection of dashboards out of the box, and is flexible enough to easily roll your own metrics. You can try our free forever account to get started.

Need help?

Our team is here for you. Whether you’re new to monitoring or ready to take your dashboards to the next level, we’ll help you get the insights you need—no jargon, just clear answers. Head here.

This blog was last updated on June 6, 2025