In a sense, deployments are the most crucial part of the CI/CD pipeline. If deployments don’t happen, the CI/CD pipeline may as well not exist at all. Nonetheless, deployments tend to be among the CI/CD processes that developers monitor the least closely.
Think about it: You probably analyze all sorts of metrics and logs from your production applications and host environment. How much data do you collect about the deployments that put applications into those host environments in the first place? Probably very little.
That’s a problem because a lack of visibility into deployment patterns makes it harder to manage things like rollbacks and ensure that your CI/CD pipeline is moving at just the right speed – not too fast and not too slow.
Fortunately, you can gain insight into deployments. But doing so requires thinking beyond the conventional application and OS logs and metrics that teams usually emphasize. Instead, developers must think more broadly and analyze the pipeline itself.
Why Track Deployments Through CI/CD Logs
Again, the typical developer probably spends much more time and energy obsessing over data from production environments than thinking about deployments. That’s a little strange when you think about it, as without deployments, there would be no production environment in the first place (or at least, there would be no production environment with a production application running in it).
More specifically, without insight into data such as the frequency of deployments, average time to deploy, and more, it is much harder to perform several critical tasks, such as:
- Tracking pipeline speed: How often do you need to release new application versions to support business objectives? Are you meeting those numbers? Without data on deployments, it’s hard to know.
- Tracking SLA compliance: In some cases, teams may be beholden to Service Level Agreements, or SLAs, that govern how quickly they will update software or fix issues. Tracking CI/CD deployments provides continuous insight into how well positioned the team is to meet SLA requirements.
- Performing rollbacks: If you decide it’s necessary to roll back an application release to address a critical security or performance issue, you’ll want to know how long to expect the redeployment of an earlier version to take.Optimizing deployment frequency: Along similar lines, your CI/CD pipeline may be moving too fast. For example, you may be making new deployments where each deployment contains only a handful of changes, leading to high deployment frequency but low impact per deployment. In that case, it may be healthier to slow the pipeline down a bit and consolidate multiple deployments into one.
- Managing configuration drift: Tracking data like average deployment time is one way of catching issues related to configuration drift (which means small changes to configuration files over time), leading to inefficiencies in the deployment process.
Tracking deployment health by logging CI/CD data can be helpful for a variety of other purposes as well. Above, we’ve noted just some of the most important.
5 CI/CD Metrics to Log for Deployment Analytics
Now that we know why it’s essential to log and analyze data that helps you understand deployment health, let’s look at which CI/CD metrics to log and analyze toward that end.
Deployment Frequency
Deployment frequency tells you how many deployments you achieve per day or per week (or per month, if you have a relatively slow CI/CD pipeline). Again, you can use this data to ensure that your deployments are as efficient as possible and reinforce business goals.
Time-to-Deploy
How long does it take to push an application release that has passed testing into production? How long does it take to decommission the earlier version of the production app? The answer will depend on how many environments you have to deploy to and which type of infrastructure you deploy (deploying applications through a managed cloud service, for instance, is usually faster than deploying to basic IaaS VMs).
But whatever your case may be, tracking the Mean Time To Deploy (MTTD) is vital for anticipating how long a rollback may take and identifying points where your deployment times may be slowing down due to problems with your CI/CD tools or configurations.
Failed Deployments
How often does a deployment fail due to incompatibility between the staging environment and the production environment? Ideally, the answer is never, but you should know either way so that you gain early insight into issues that could lead to failed deployments. After all, an automated processes means little if you can’t deploy reliably. Failure rates are a critical metric to track.
Lines of Code per Deployment
Where feasible (in other words, when your CI server or SCM tools make it practical to log this data), consider tracking how many lines of code you include in each new deployment. Although this number may fluctuate considerably since the amount of code required to implement a new feature can vary widely, you generally want to avoid situations where each deployment contains very little new code or a considerable amount of new code.
In the former case, you may be deploying more often than necessary, and you could make your CI/CD pipeline more efficient by merging some application releases. In the latter, you may want to consider deploying more frequently. Finding that balance requires reliable measurements.
Rollback Frequency
Deployment issues are not the leading cause of a rollback (security or performance bugs usually are). Still, deployment problems like misconfigurations or incompatibility issues between a new release and the production environment could lead to rollbacks. For that reason, tracking the frequency of rollbacks is another way to gain some insight into your deployments.
On top of this, tracking rollbacks via logs is also useful for helping teams to identify which root-cause issues most frequently lead to rollbacks. In turn, they can improve their CI/CD processes to minimize the need to perform rollbacks in the first place.
Conclusion: Logging Across the Pipeline to Track Deployment Health
While it may seem strange to think of data about CI/CD processes like deployment as a vital part of a logging and analytics strategy, the fact is that deployment patterns are an essential piece to understanding the health of both your application and infrastructure.
By logging and analyzing deployment operations, teams put themselves in the most advantageous position to optimize the overall CI/CD pipeline and ensure that deployments don’t become the weakest link in the CI/CD process.This also gives a sense of security in knowing that your systems can be reverted to the most stable version quickly, no matter how devastating the outage.
