# Change Failure Rate

{% hint style="info" %}
⭐️ This metric is one of the [4 Key Metrics published by Google's DevOps Research and Assessment (DORA)](https://dora.dev/guides/dora-metrics-four-keys/) team. 

You can see all 4 key DORA metrics on the [DORA Metrics page](https://app.multitudes.co/DORA) of the Multitudes app. 
{% endhint %}

{% hint style="warning" %}
Note: this metric is **only shown at a team level**, not an individual level.
{% endhint %}

![Change Failure Rate graph](https://cdn.prod.website-files.com/610c8a14b4df1ae46b1a13a3/638c21ca0787c76083dcc4e2_Change%20Failure%20Rate.png)

**What it is:** The percentage of PRs merged that indicate that some kind of failure was released and had to be fixed.

**Why it matters:** This is our take on [DORA's Change Failure Rate](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance), which indicates the percentage of deployments that cause a failure in production. It's a lagging indicator of the quality of software that is being deployed - how often does it contain bugs that cause failures later?

**How we calculate it:** We calculate the % of merged PRs that contain any of the words `rollback`, `hotfix`, `revert`, or `[cfr]` in the PR title, out of all merged PRs. We tested and chose these keywords to catch most cases where a PR was required to fix a previously-released bug, while minimizing false positives.

<details>

<summary>Tip: Use a script to automatically add [cfr] to PRs that meet certain criteria</summary>

In the future we will allow customization of which key words we take into account when determining what constitutes a change failure. In the interim if your team has established practises around naming conventions, you can use a CI script to add `[cfr]` to the title of PRs you wish to track in Multitudes. This example Github Actions job checks to see if the PR title contains the word `fix`, and if so appends `[cfr]` to the title.&#x20;

```
name: Multitudes - Change Failure Rate

on:
    pull_request:
        types: [opened, edited] # runs when PR is created or the title is changed

permissions:
    pull-requests: write
    issues: write

jobs:
    update-title:
        runs-on: ubuntu-latest
        steps:
            - name: Adjust title if necessary
              uses: actions/github-script@v7
              with:
                  script: |
                      const pr = context.payload.pull_request;
                      const originalTitle = pr.title;

                      // Rules
                      const hasFixPrefix = /^fix\(/.test(originalTitle);
                      const hasCfrSuffix = /\s*\[cfr\]$/.test(originalTitle);

                      let newTitle = null;
                      let comment = null;

                      if (hasFixPrefix && !hasCfrSuffix) {
                        newTitle = `${originalTitle} [cfr]`;
                        comment = "🤖 **Automated update**: Added `[cfr]` tag to PR title since this is a fix commit. This will help track 'change failure rate' in Multitudes.";
                      } else if (!hasFixPrefix && hasCfrSuffix) {
                        newTitle = originalTitle.replace(/\s*\[cfr\]$/, '');
                        comment = "🤖 **Automated update**: Removed `[cfr]` tag from PR title since this is not a fix commit. The `[cfr]` tag should only be used for fix commits.";
                      }

                      if (newTitle === null) {
                        core.info("No update needed.");
                      }

                      if (newTitle !== null) {
                        await github.rest.pulls.update({
                          owner: context.repo.owner,
                          repo: context.repo.repo,
                          pull_number: pr.number,
                          title: newTitle
                        });

                        await github.rest.issues.createComment({
                          owner: context.repo.owner,
                          repo: context.repo.repo,
                          issue_number: pr.number,
                          body: comment
                        });

                        core.info(`Updated PR title: "${originalTitle}" -> "${newTitle}"`);
                      }

```

</details>

{% hint style="info" %}
We recognise that this proxies failures after the fact; this is because it's not actually possible to know if someone's releasing a failure into production in the moment, otherwise it wouldn't have been released in the first place!

Also, incidents are not always easily tied to a specific PR or deployment. You can include the square-bracketed keyword `[cfr]` in your PR titles if you'd like more granular control over what gets counted in this chart.
{% endhint %}

{% hint style="success" %}
**What good looks like**

In Google's DORA research, the top 15th percentile have a  `Change Failure Rate` of 0-4% [(see DORA 2025](https://dora.dev/research/2025/)). The DORA benchmark has changed over the years – it was[ 0%-5%](https://storage.googleapis.com/gweb-cloudblog-publish/images/1_-_elite_performer_graph.max-1400x1400.png) until recently, and [0%-15%](https://storage.googleapis.com/gweb-cloudblog-publish/images/Calculating_the_metrics_frOhcbp.max-2800x2800.jpg) before that.
{% endhint %}