Metrics provide visibility into engineering operations, allowing teams to iteratively improve. By tracking key data points like deployment frequency, lead time, server uptime, and change fail rate over time, organizations can spot trends, compare against benchmarks, and address process inefficiencies.
The DevOps Research and Assessment (DORA), a program now owned and maintained by Google Cloud, advises engineering teams on the most essential metrics to measure and track to evaluate and improve the performance. One of these is deployment frequency, which measures how often a team releases code to production. For modern engineering teams, accelerating software delivery speed is important for both improving development processes and providing value to users. In this article, we will discuss deployment frequency, its benefits, and how to start improving it on your team.
What is deployment frequency?
DORA defines deployment frequency as “how often an organization successfully releases to production.” It encapsulates the speed and agility of a development team, serving as a key indicator of their velocity in delivering code changes to production.
Specifically, it refers to the number of software releases over a given period of time, usually measured in releases per day, week, or month. Some examples of deployment frequency metrics are 10 deploys per day, 5 deploys per week, or 2 deploys per month, however it’s best to refer to the frequency in higher level cadences such as daily, weekly, or monthly.
Deployment frequency provides insights into how quickly development teams are able to develop, test, and ship code changes. Higher deployment frequencies are associated with increased agility, faster delivery of new features and fixes to customers, tighter feedback loops, and overall better alignment with business goals. It is often considered a strong indication of the maturity of both the system and the development team. Deployment frequency represents overall system health because it's the result of many different upstream components like engineering culture, automated infrastructure, and systems.
Benefits of tracking deployment frequency
Tracking deployment frequency provides valuable insights into a development process. By monitoring the release rate of code changes, teams can identify patterns and trends, gauging the efficiency of their deployment pipeline. This includes the team's responsiveness to user feedback, detection of bottlenecks, and an overall assessment of the development workflow's health. Conversely, issues like prolonged lead times, high error rates, or inconsistent deployment patterns can be swiftly identified, paving the way for targeted improvements. This process also uncovers opportunities for optimizations, highlighting trends that suggest areas for increased automation, streamlined workflows, or more efficient collaboration among team members. We’ll discuss the benefits in detail below.
- Faster delivery of features and fixes: Monitoring and improving deployment frequency ultimately leads to more features released to production faster. Teams can provide value to customers sooner.
- Continuous improvement: Regularly tracking deployment frequency fosters a culture of continuous improvement, where teams are driven to refine and optimize their processes.
- Early detection of issues: Development teams that deploy changes rapidly and consistently can quickly detect any anomalies, bugs, or unexpected behavior that may arise due to code changes. By addressing concerns in the early stages of deployment, teams not only reduce the potential impact on users but also streamline the debugging and troubleshooting processes.
- Better communication and collaboration: The requirement for frequent deployments necessitates efficient communication and collaboration among team members for coordinating tasks, sharing insights, and swiftly addressing challenges. The emphasis on regular collaboration ensures that team members stay well-informed about ongoing developments among various teams and projects.
- Risk reduction: Smaller deployments make it easier to pinpoint issues, such as bugs or performance issues, as they emerge. This granularity allows development teams to address these challenges swiftly, preventing them from accumulating and escalating into more significant problems during a large-scale release.
- Faster feedback loops: By swiftly pushing changes into the live environment, development teams can promptly collect valuable insights and feedback based on real-world usage of the software. This immediacy in gathering user input enables teams to iterate rapidly, fine-tuning features, addressing user concerns, and making improvements based on tangible experiences.
- Increased predictability: More predictable deployments are invaluable in project planning, allowing teams to anticipate and align development milestones with a dependable deployment schedule. With a known frequency of deployments, stakeholders can make informed decisions about resource allocation, release schedules, and coordination with other project-related activities. This enhanced predictability not only streamlines project planning but also fosters a more reliable framework for meeting deadlines and ensuring the smooth progression of development initiatives.
- Competitive advantage: A high deployment frequency provides a competitive edge, allowing organizations to outpace competitors and respond swiftly to market dynamics.
- Compliance and security: Continuous deployment allows development teams to quickly address emerging security concerns, deploying patches and updates as soon as they are available. This proactive stance not only helps in staying ahead of potential vulnerabilities but also aligns with regulatory demands that mandate prompt action in the face of security risks.
- Improved customer experience: When deploys are frequent, user issues and requests can be addressed more promptly. A commitment to quality through frequent deployments directly translates into an enhanced customer experience, building trust and satisfaction.
How can I calculate deployment frequency?
Understanding the calculation of deployment frequency is fundamental to its effective use. The process is straightforward, encompassing every type of code change, whether it be a substantial new feature or a quick bug fix. The formula is simple: the number of change deployments divided by time equals deployments per unit of time. Teams can choose to measure deployment frequency on a daily or weekly basis.
Number of deployments / time = Deployments per unit of time
For example, if a team makes 20 deployments in a week, their weekly deployment frequency is 20. You can also aggregate the metrics to make comparisons easier. In another example, imagine you want to measure deployments by the number per day, but compare those between weeks, you can take the average for the week and compare it to other weekly averages. If you measure a team’s deployment frequency over time, trends and patterns start to emerge.
What is a “good” deployment frequency?
A high deployment frequency typically signals a healthy production environment. However, accurately gauging whether your deployment frequency is "good" or "bad” requires consideration of multiple aspects. Several factors, such as the complexity of the application, team size, and the degree of automation in place, exert substantial influence and require careful examination.
The latest State o DevOps report from DORA sets four benchmark categories of team performance: elite, high, medium, and low. Elite teams deploy on demand. The high-performing teams deploy once per day and once per week, while medium and low-performing teams deploy between once per week and once per month. In general, we can define a good deployment frequency as more than once per week, with an optimal frequency as multiple times a day.
When tracking and creating targets for this metric, it’s important to categorize the deployment frequency into larger buckets like the ones provided by DORA, instead of being meticulous about the actual number of deployments. Choose a goal like “multiple times a week” as opposed to three times a week to avoid falling victim to Goodhart's Law: when a measure becomes a target, it ceases to be a good measure. The aim is not to hit specific numbers but rather to create a deployment system that’s fast, reliable, and flexible.
Establishing realistic expectations based on the specific context is necessary for deriving any meaningful insights from deployment frequency data. Although on-demand deployment is the gold standard, a moderate and steady deployment frequency might prove more suitable in some scenarios, especially if it aligns with the complexities and demands of the application being developed.
How to improve deployment frequency?
The end goal of all software development boils down to one key outcome: delivering value to end users. The faster we can develop new features, enhancements, and fixes and deploy them into production, the quicker we generate value. While maintaining care to not sacrifice code and product quality, there are a number of ways teams can look to increase their deployment frequency.
- Automation: Repeated manual tasks during deployments invariably slow things down and introduce risk. By leveraging DevOps automation tools or scripting deployments end-to-end with infrastructure-as-code and CI/CD pipelines, teams remove friction through reliable automation. Standardized scripts also enhance consistency across environments.
- Implement CI/CD: Continuous integration and continuous delivery (CI/CD) is necessary to attain on-demand deployment and is one of the most important systems for achieving significant improvements to deployment frequency. At its core, CI/CD is a methodology that ensures a seamless and automated flow from code development to production deployment. By automating the integration of code changes into a shared repository (continuous integration) and subsequently automating the deployment of validated code to production (continuous delivery), development teams can significantly reduce the likelihood of integration issues and streamline the delivery pipeline, as well as save engineers lots of time on manual steps.
- Test-driven development (TDD): Test-driven development, where engineers write tests ahead of writing code, produces more robust systems. With greater confidence in code behavior, teams can deploy faster without jeopardizing quality. Catching issues earlier in the development process can prevent failed deployments, which are slow and resource costly.
- Feature flags: In contrast to large bundled releases, feature flags allow incremental deployments and controlled activation in production. Flags decouple the code release from the feature release, so code can be deployed when it’s ready. This reduces risk and complexity when launching new features while enabling more frequent releases, and it prevents code from getting stale and further slowing the deployment process.
- Feedback loops: Tighter feedback loops and a culture of constructive criticism enable teams to adjust deployment processes based on findings and past experience. You can close the loop with developers by providing fast and tangible feedback on their code's production performance, focusing on improving cycle time and reliability This high visibility motivates higher code quality and safety standards uplifting velocity.
- Capacity planning: Capacity planning contributes to the improvement of deployment frequency by fostering a streamlined and efficient development environment. It allows for actively identifying and addressing potential bottlenecks before they occur. By ensuring a steady flow through the development stages, teams can maintain a consistent and predictable deployment frequency.
- Documentation and knowledge sharing: Optimal deployment practices involve empowering all engineers, not just those with specific expertise, to participate in the deployment process. Teams can foster a collaborative and informed environment by thoroughly documenting processes and promoting knowledge sharing. Ensuring that the deployment process is easily comprehensible and replicable enhances the capability of a broader group of engineers within the team to participate in deployments and accelerates the overall deployment speed.
- Education and training: Similar to knowledge sharing, investing in educating your engineers on how the current deployment system works and methods that can be used to improve it can also prove beneficial. This allows them to deploy more efficiently and independently.
- Smaller, more frequent deployments: Releasing incremental updates more often instead of larger deployments reduces complexity and risk. Less complex deployments are more likely to complete successfully, enabling engineers to deploy more confidently and therefore more often.
- Set targets, monitor, and measure: It’s difficult to improve what you can’t measure. By implementing robust monitoring and measurement practices, you can track the impact of changes and identify areas for further optimization. Once you understand the current state of your deployment frequency, you can define achievable targets based on the team's capacity and the project's requirements.
Some of these practices may already exist in your team’s workflow, while others may not apply to your situation. Determine which are appropriate for you to implement or improve on to improve deployment frequency for your team.
Other engineering metrics to track
We’ve discussed in detail the importance of tracking deployment frequency, but it’s only one metric of many that are necessary to develop a complete picture of the state and efficiency of an engineering team. When considering which metrics to track, consider starting with the other DORA metrics in addition to deployment frequency:
Lead time for changes (LTC)
Failed deployment recovery time (FDRT)
Change failure rate (CFR)
Reliability
Tracking these metrics together, according to DORA, provides “a valid and reliable way to measure software delivery performance.” Combined they give insight into the speed, quality, and consistency of releases. Understanding and tracking metrics as a cohesive set enables teams to make informed decisions that positively impact both short-term goals and long-term strategic objectives. If you want to learn more metrics you can track and how to choose which to track, check out our pocket guide to engineering metrics.
How Cortex connects productivity metrics to overall software and team health
To make both team and software performance metrics actually useful, they need to be evaluated in context. How does software health impact productivity? How does the complexity of projects owned by a given group impact their velocity compared to a newer team with fewer services to maintain? Is deployment velocity up for teams with the healthiest software? Is it down for those that have a number of vulnerabilities to patch due to lapsed security integrations? Internal Developer Portals (IDPs) like Cortex provide all of the context about your development environment you need to make better decisions with data. They also help track, improve, and accelerate the development of high quality software by abstracting away the complexities of ensuring software security, maturity, production readiness, and more—all using data from your existing tools.
Unlike other IDPs, Cortex also offers a way to collate and track productivity metrics that can enrich or be enriched by information about software health captured via Scorecards and Initiatives. Eng Intelligence in Cortex provides comprehensive metric tracking over time by team or group so you can surface trends in git, jira, pagerduty, or deployment data to help improve resourcing, or set new standards.
Cortex Scorecards can also be used to set goals for these and other critical standards of software or team health, including DORA metrics. Use data from any integrated SDLC software like Kubernetes, AWS, and OpsGenie, to set numerical or binary rules according to standards you defineTo learn more about how tracking software health relates to productivity metrics, check out this blog, or watch our on-demand webinar on developer productivity.
Interested in learning more about how Cortex can help you monitor your deployment frequency and other DORA metrics? Check out our product tour or schedule a demo today.
