Home/Services/DevOps & DevSecOps/Azure DevOps Health & Modernization

Services

Azure DevOps Health
& Modernization

Short answer

Azure DevOps modernization is the work of finding where an Azure DevOps organization has drifted (classic pipelines, broad permissions, stale service connections, fragile agent pools) and fixing it in code so the org can be rebuilt and audited. Mentat starts with evidence: the free ADO Health Check CLI or a 60–90 minute assessment. Then we work through a ranked backlog with your team instead of handing over a binder.

Most Azure DevOps organizations were not designed. They accumulated. A project per team, a pipeline per app, a service connection somebody created in a hurry, and a self-hosted agent under a desk that nobody wants to touch. It works, until an audit, an outage, or a migration asks who can change what.

Run the free ADO Health Check

Who it is for

Teams that run on Azure DevOps and no longer trust what is in it

This work fits organizations where Azure DevOps is the delivery backbone and the people who built it have moved on, multiplied, or are simply out of time. It is a good fit if any of these sound familiar:

  • Platform or DevOps leads who inherited an org with dozens of projects and no clear owner for half of them.
  • Engineering managers who need delivery to keep moving while security asks for evidence about pipelines and access.
  • Regulated teams, including government programs, that have to show how a change reached production and who approved it.
  • Teams planning a move from classic release pipelines to YAML, or consolidating several organizations or projects, who want to know what will break first.
  • Leaders who want to add automation or AI agents to delivery and know the foundation has to hold the weight first.

Problems it solves

What an unhealthy Azure DevOps org looks like

None of these are exotic. They are the normal residue of a busy org, and each one turns into risk quietly.

  • Classic build and release pipelines that live only in the UI, so nobody can review a change to them or rebuild them from Git.
  • Service connections with more scope than they need, shared across projects, authenticated with secrets that never rotate.
  • Variable groups holding secrets in plain values instead of being marked secret or backed by Azure Key Vault.
  • Project Collection Administrators and project admins that grew one favor at a time, plus guest accounts nobody remembers inviting.
  • Branch policies that are missing, inconsistent, or bypassed, so "required review" means different things in different repositories.
  • Self-hosted agent pools that are a single point of failure, running mismatched agent versions on machines nobody patches.
  • Retention and failed-run trends nobody watches, so a pipeline that fails one run in five is treated as normal.

Our approach

Measure first, fix in code, keep the team in the loop

We do not start with a reorganization diagram. We start with an inventory of what is actually there, because the gap between what people believe about an Azure DevOps org and what the API returns is usually where the risk lives.

From there the rule is simple: every fix should leave the org more reproducible than we found it. Pipelines move into YAML templates in Git. Permissions move toward groups and least privilege. Service connections move toward workload identity federation where Azure supports it, so there is no secret to rotate. Agents become infrastructure as code instead of pets. Policy checks run in the pipeline, not in a wiki page.

We work alongside the people who own the org. The point is not to make you dependent on Mentat. It is to leave a platform team that can explain every pipeline and every permission, and a paved road that makes the secure way the easy way. That is the same thinking behind our platform engineering and DevSecOps practices, applied to the tool most Microsoft shops already run.

Process

How an engagement runs

1. Scan

Run the free ADO Health Check CLI against your organization, locally or in an Azure Pipelines job. It produces a JSON inventory and an HTML report covering pipeline hygiene, identity and access, and agent and pool posture. Nothing leaves your environment unless you choose to share the report.

2. Assess

A 60–90 minute working session with the people who own your pipelines. We walk the findings, score maturity across delivery, identity and secrets, drift, and observability, and agree on 3–5 ranked next moves.

3. Scope

Turn the next moves into a backlog with owners, order, and a definition of done for each item. Quick wins such as secret variables, stale admins, and orphaned service connections go first because they shrink risk immediately.

4. Modernize

Work the backlog with your team: YAML pipeline templates, branch policies, service connection cleanup, agent pools as code, and promotion gates that a reviewer can follow. Changes ship as pull requests, so your history explains itself.

5. Hand over and re-check

Re-run the health check to show what moved, document the operating model, and leave the templates and policies with the team that will own them.

Deliverables

What you leave with

  • A baseline health report and inventory of projects, pipelines, service connections, variable groups, and agent pools.
  • A maturity scorecard and a ranked remediation backlog with owners.
  • Reusable YAML pipeline templates and a migration path off classic pipelines.
  • A permissions and service connection model built on groups and least privilege, with the exceptions written down.
  • Agent pool definitions in code, with a plan for Microsoft-hosted versus self-hosted capacity.
  • A follow-up health check that shows the before and after, measured the same way.

Timeline

How long it takes

The health check takes minutes to run. The assessment is a single 60–90 minute session. Remediation depends on the size of the org and how much of it is classic, so we scope it after we have seen the report rather than guessing up front.

We usually split the work into phases. The first phase tackles the access and secrets findings that carry the most risk. Later phases cover pipeline migration and agent work, project by project, so delivery never has to stop for a big-bang cutover.

Experience

Azure DevOps at real scale

Mentat has used Azure DevOps as the orchestration layer on large programs, including a published government automation platform serving more than 100,000 users that combined Azure DevOps orchestration, Azure Kubernetes Service hosted agents, Terraform, and Ansible. The details are in the case study. Results on one program are not a forecast for yours, which is exactly why we start by measuring.

100,000-user automation case study12 checks for a healthy Azure DevOps org

Related

Continue exploring Mentat

ADO Health Check CLIHow to run the health checkCloud Automation AssessmentDevOps & DevSecOpsPlatform EngineeringAzure Landing Zones12 Azure DevOps health checks

Frequently asked questions

Questions about Azure DevOps health and modernization

What is Azure DevOps modernization?

Azure DevOps modernization is fixing the drift that builds up in an Azure DevOps organization over time: moving classic build and release pipelines to YAML in Git, tightening permissions and service connections to least privilege, moving secrets into secret variables or Azure Key Vault, and defining agent pools as code. The goal is an org you can review, rebuild, and audit, not a migration for its own sake.

How do I know if my Azure DevOps organization needs work?

Run the free Mentat ADO Health Check CLI at https://www.mentat.cloud/tools/ado-health-check. It writes a JSON inventory and an HTML report on pipeline hygiene, identity and access, and agent and pool posture. Common warning signs are classic pipelines nobody can rebuild, broad Project Collection Administrator membership, shared service connections with long-lived secrets, and a self-hosted agent pool that is a single point of failure.

Do we have to stop shipping while Azure DevOps is modernized?

No. Mentat works project by project and phase by phase. Access and secrets fixes usually go first because they reduce risk fastest, then pipeline migration and agent work follow without a big-bang cutover. Changes ship as pull requests your team reviews.

Does Mentat migrate classic pipelines to YAML?

Yes. Mentat builds reusable YAML pipeline templates, moves classic build and release definitions onto them, and adds branch policies and promotion gates so pipeline changes are reviewed like any other code.

Is Azure DevOps modernization the same as moving to GitHub?

No. Modernization here means making your existing Azure DevOps organization healthy, reproducible, and secure. If a move to another platform is on the table, the same inventory and cleanup make that move smaller and safer, and Mentat will say plainly if it does not think a move is worth it.

How long does an Azure DevOps health check and modernization take?

The ADO Health Check CLI runs in minutes and the Mentat assessment is a single 60–90 minute session. Remediation is scoped after the report, because it depends on how many projects, classic pipelines, and agent pools are in the org. Book at https://www.mentat.cloud/assessment.

Start with evidence

Scan your org for free, or book the assessment.

Run the ADO Health Check CLI yourself, or book a 60–90 minute assessment and we will walk the findings with your team and rank the next moves.

Get the ADO Health Check