6 Design Principles for Azure Pipelines

September 26, 2024

Read Time: ~10 minutes

As companies scale, system pressures increase, resulting in challenges like code conflicts and maintenance complexities. At Empower, our monorepo architecture led to significant maintenance overhead due to the tight coupling of build and deployment processes across services. To address this, we restructured our deployment system by decoupling build from deployment, enabling independent service deployments. This post outlines the key principles we developed to maintain clarity and efficiency in our Azure pipelines.

As companies grow, pressure inevitably builds on different parts of their systems. With more customers, certain code paths start to see increased traffic, and developers begin to overlap in unexpected areas, leading to code conflicts.

Empower has recently faced these challenges in its delivery system. Our backend is organized as a monorepo, meaning nearly all backend components are housed within a single repository. Among these components is a service used by our support team. This service shares a data model with our main API, which means we have to version and deploy both of them together. As teams began needing to deploy these services separately, we encountered maintenance challenges due to coupling of our build and deployment implementations across these services.

We decided to restructure our deployment system to address the general maintenance overhead using a new model. This model was discussed in a previously post - which you can check out if you're interested. In summary - the model was to build the mono-repo continuously, and have the build be decoupled from deployments. So all backend services would be built together, but they would be deployed independently.

Throughout this process of refactoring and rebuilding, we identified and established several key principles to follow to keep our pipelines comprehendible and maintainable. I'd like to share these principles and associated insights with you in this post.

Principles for Developing Deployment Systems in Azure

The following are 6 core principles you can follow to achieve success in your Azure pipeline implementations.

1. Decouple Build from Deploy

This is more of a reiteration of the general model described in the previous post, but it’s worth enshrining as a principle: Do not pollute deployment pipelines with build pipelines. Doing so keeps maintenance complexity down and prevents build overhead during deployments, making them much faster.

In times when you need to redeploy quickly, having to wait for a build is a significant time sink. Also - when delivering software, we should always aim to follow a ‘build-and-publish-once’ philosophy. Once your software is built and published - that is the final version prepared for a release. You can test it multiple times, you can deploy it multiple times, and you can promote it through environments - but you should only ever build and publish it once.

2. Design at the Job Level

Pipelines, whether single or multi-stage, are nothing more than a collection of jobs that execute in some sequence. Jobs represent all of the actions your pipeline will take while on a given agent (i.e., worker computer). Having the ability to define distinct jobs allows for things like parallel execution and encapsulation of tasks.

Jobs should be meaningful encapsulations of components of your build or deployment pipelines.

Consider the following job:

jobs:
  - template: ../jobs/CalculateVersionWithReleaseBranchOption.yml
    parameters:
      jobName: SetVersion
      versionSetStepName: SetVersionStep
      versionVariableName: AppVersion
      releaseBranchName: ${{ variables.releaseBranchName }}
      bumpMinor: True
      variableGroupName: ${{ variables.mainlineVersionTrackingName }}
      variableGroupKeyId: ${{ variables.mainlineVersionTrackingId }}
      devopsUrl: ${{ variables.devopsUrl }}

It should be clear from the name of this job template that we have encapsulated all of the concerns of calculating a release version in this job. This keeps this job’s purpose plain and understandable for maintainers reviewing the design of the pipeline, and ensures that we can re-use this template in any pipeline that requires version calculation.

3. Think of Templates as Functions

For any given job, there are likely logical groupings of behaviors. These related behaviors should be encapsulated within template files - not just for reusability, but also for clarity of purpose.

Leveraging templates as functions solves this problem by providing, essentially, a label to a logical grouping of behavior. Take for example the following template used to invoke dotnet test:

parameters:
  - name: settingsFilePath
    type: string
  - name: filterString
    type: string
  - name: displayName
    type: string
  - name: testTargetCsprojOrDll
    type: string
  - name: retryCount
    type: string
    default: 0
steps:
  - task: DotNetCoreCLI@2
    displayName: "Status Check: ${{ parameters.displayName }}"
    retryCountOnTaskFailure: ${{ parameters.retryCount }}
    inputs:
      command: test
      projects: ""
      publishTestResults: false
      arguments: |
        ${{ parameters.testTargetCsprojOrDll }} 
        --filter ${{ parameters.filterString }} 
        -d ${{ parameters.displayName }}Logs.$(Build.BuildId).txt
        --configuration Release
        --logger trx;verbosity=detailed
        --results-directory ${{ parameters.displayName }}Results.$(Build.BuildId)
        --collect "XPlat Code Coverage"
        --settings ${{ parameters.settingsFilePath }}
  - task: PublishTestResults@2
    displayName: "Post: Publish Test Results"
    condition: succeededOrFailed()
    inputs:
      testResultsFormat: VSTest
      testResultsFiles: "*.trx"
      searchFolder: ${{ parameters.displayName }}Results.$(Build.BuildId)
      mergeTestResults: true
      buildConfiguration: Release
      publishRunAttachments: true
      failTaskOnFailedTests: true
      publish: ${{ parameters.displayName }}Results.$(Build.Build.Id)
      artifact: ${{ parameters.displayName }}Results.$(Build.Build.Id)

This template groups together behaviors for executing tests as well as publishing the results. It does so in a way that generalizes the function outputs which are artifacts. These behaviors will always move together, and if I absolutely must disable publishing for some reason, I can introduce a parameter to facilitate that.

4. Parameterize Values that are not Intrinsic to the Template

When designing the boundaries of a given template, you will frequently need to make judgment calls on what is a concern or responsibility of a template, and what is not. A decent rule of thumb to follow for this is: behaviors are intrinsic, details are not.

5. Parameterize Values that are Likely to Change

Certain variables - when loaded into a pipeline - can be accessed from anywhere. Consider the following variable group, which defines a variable MyVariable, which is then accessed:

name: MyPipeline
variables:
  - group: MyLibraryVariableGroupInADO
jobs:
  - job:
      steps:
        - template: ./MyTemplate.yml
          parameters:
            someVariable: $(MyVariable)

6. Use a Global Variable Template

The final principle I’d like to share in this post is not without some contradiction to advice against introducing magic - however it’s been found to be a major benefit in the Empower pipelines. A global variable template is simply a central place to provide access to a common set of variables that may be used by one or more pipelines.

global-variables.yml
variables:
  # agents
  - name: linuxVmPool
    value: linuxAgent
  # service account PAT
  - group: CommonPipelineAccessTokens
    - name: githubRestApiAccessToken
      value: $(GitHubPAT)

This can be imported into any pipeline using:

variables:
  - template: ../../global-library-variables.yml

About Paul Gradie

Paul is a software engineering leader / data scientist / biologist / team player / father with published contributions in the fields of reproductive biology, artificial intelligence, and software engineering.

Paul enjoys pair programming, learning, and building things well and is often told that he is personable, approachable, and enjoyable to work with.