LogDNA and CI/CD

                                                                                              Source: https://opsani.com/resources/what-is-ci-cd/

After reading this article, you will be able to answer the following questions:

• What is CI/CD? 
• How can centralized logging help me better understand my CI/CD?
• Which monitors and metrics can help me with logging implementation? 

Continuous Integration/Continuous Delivery

What Is CI/CD?

As an engineer at any organization, you will encounter CI/CD pipelines. These pipelines enable applications to be updated by deploying the latest code changes which are then manually or automatically deployed to all of your environments. The “CI” in CI/CD stands for Continuous Integration; put simply, this is the practice of automating the integration of code changes from multiple contributors into a single project (which is typically a shared GitHub repository). The “CD” component stands for Continuous Delivery, which equates to high degrees of automation that can help push code into production faster than ever. CI/CD is designed to require less human intervention, since there’s a chance of human error each time humans are involved in a process. By using CI/CD, you can ensure that you have more secure and more efficient pipelines that can be frequently changed or updated on the fly. In order for this to run smoothly and not break everything in sight, testing must be part of your release cycle.  

CI/CD Tooling

Even if you are not a “DevOps engineer,” you will still interact with your organization's CI/CD. Fortunately, there are many tools that can help you. For example, your organization may be using applications like:

• Jenkins https://www.jenkins.io/
• Bamboo https://www.atlassian.com/software/bamboo
• GitLab https://about.gitlab.com/
• CodeShip https://www.cloudbees.com/codeship/
• CircleCI  https://circleci.com/ 
  

CI/CD pipeline tools enable you to automate your software delivery process. Without them, you would spend most days running several commands in order to apply your code changes to the applications that your company manages.

When using your CI/CD tool of choice, it's important to remember that these systems are predominantly gated. This means that a feature or release candidate is first released into a lower environment, then run through testing. Finally, a team will either deny or approve the process for deployment into the next environment until it reaches production.

Centralized Logging

Centralized Logging with Mezmo

Mezmo, formerly LogDNA, is different from other log management tools in that it aggregates all system and application logs into one centralized system while also using automatic parsing and intelligent filters to efficiently query the log lines you need.

How to Set up Centralized Logging with Mezmo

Mezmo is very simple and fast to set up. You’ll be able to gather, monitor, alert, parse, live tail, graph, and analyze your logs is a few simple steps:

1. Set up an account here.
2. Select your preferred log ingestion method.
3. Search your logs.
4. Create Views and attach Alerts.
5. Create Boards and Graphs.
   

For a speedy setup, check out the Quick Start Guide. Below, we’ll dive into each of these steps in a bit more detail.

Set up an Account

You can set up an account simply by going to the Mezmo sign up page. You will automatically be enrolled in a 14-day free trial.  After you’ve set up your account, you’ll be asked for your organization's name. Then, you will have a few different methods for starting your log ingestion. 

Log Ingestion Methods

After you have set up your account, you’ll need to decide which log ingestion method will work best for you. You’ll be provided with an ingestion key as a way to connect, and you’ll have several options:

• Use the provided ingestion key
• Install the Agent
• Install the integrations for your platform(s)
• Use Syslog
• Use code libraries

CI/CD

To show you how it all works, we will use CircleCI as our CI/CD tooling of choice. From the UI, we can see that the build has failed. We don’t quite know why it failed, so let's set up logging using Mezmo to dig deeper. 

Setting up Mezmo for CircleCI Tool

You’ll need to have version 2.1 of CircleCI, so be sure to check your config file and update if necessary. From here, you can search for the Mezmo Orb. If you’d like more detail before we begin, you can review this page. 

What Is an Orb?

An orb is a sharable package of CircleCI configuration that you can use for your builds. You can choose from a public registry or you can create a private orb that can better suit your needs.

Let's set this up so that we can be notified of the status of a build event from our CI/CD pipeline. We’ll need some information from Mezmo to set this up. 

You will need to update your configuration file with the code snippets outlined in the next step. 

Search Your Logs

After you have successfully set up Mezmo for CircleCI, you'll be able to view and search your logs with ease. For example:

Create Views for Your Error Logs

Setting up views with your error logs is super easy! Under the “Levels” tab, you can sort by the log level. If you only select the “Error” logs, you’ll create a view that will only display your error logs.

After selecting the log level error, you can create a new view so that you only see the error logs. 

Using Logging for Your CI/CD

Managing and Measuring Your CI/CD 

Now that you’ve (probably) interacted with a CI/CD tool, let’s see how we can make it better. More specifically, let’s see how we can better measure what success looks like from the CI/CD pipeline.

One of the most important things that centralized logging can do for your pipeline is to give you the ability to watch changes being applied to data as it flows in each environment, from development, to non-production, to production environments. 

If you don’t have logging set up for your CI/CD pipeline, you won’t be able to see how changes affect the environment, and it will be extremely difficult to know how to move forward on the path to a successful production release. Without logs, you won’t be able to understand the effects of your code as it moves through the pipeline, which will prevent your team from being able to confirm that your changes are acceptable to move to the next environment in your stack.

Conclusion

You probably don’t have time to watch a deployment move through your CI/CD pipeline. A logging system will allow you to see what happened at a specific time that triggered a specific alert on lower environments (“lower" meaning earlier in the pipeline). This level of insight can be a tremendous help when it comes to understanding your workflow and finding the bottlenecks in your pipeline process.