How to Organize and Tag Your Logs
Developers must have access to the information that they need. Accessing this information, however, can be time-consuming. With the wrong tools or workflow, it’s frustrating and challenging to filter logs by source.
Tags are an easy solution from Mezmo, formerly known as LogDNA, that helps developers get a more granular view of what’s going on in our services.
Logging with Tags
Tags are familiar concepts in any form of organization — and even on social media. On photo-sharing sites, users often tag people in photos. When users browse images, we can easily configure the software to display all images of a person or just two people. Tagging anything with extra data makes browsing and sorting easier and often more efficient. On Mezmo's platform, tags refer to an attribution given to a particular log or set of logs on Mezmo's platform. In this specific context, tags serve as helpful metadata about the source of a log entry. Using the Mezmo agent or Syslog-based log parsers makes adding tags easy since the source is often detected automatically.
We can apply this concept to our application logs, making critical data easy to find and browse through. We can use multiple tags in a single logline and later filter our searches or console to show only logs that contain or don’t contain that log. Logs with tags are more valuable to development teams, and Mezmo makes it easy to set them up from the start.
Tags let Mezmo automatically group log entries into host groups, a critical feature that ensures our application logs remain neatly organized. Logs from a single host can contain multiple tags, indicating more than one reference to a single host. Though right now, only source tags are available, more are on the Mezmo roadmap.
It may seem redundant to filter by source or source tag, but log tags allow for more advanced filtering that proves valuable when issues arise in specific systems. Searching in Mezmo is incredibly flexible, with simple queries available and advanced queries that can leverage these tags right at our fingertips.
It’s helpful to consider a typical scenario where we run a fleet of servers or containers. Some likely serve our web pages, others help our web API, and even more run back-end services. For our webpage servers, we might set the tags “web-server” and “webpage-server.” For our web API servers, we might assign the tags “web-server” and “API-server.” If we organize our logs this way, it’s easy to filter searches down to only what we're trying to find. Using filtering to search just webpage servers or only API servers is trivial. But with this setup, it’s just as easy to search for logs from servers that serve any HTTP(S) requests since they all use the web server. You can learn more about the particulars of assigning tags to sources in Mezmo's documentation.
The most overall utility comes from adding source tags based on the function of the application, virtual machine, or container. If we have a dozen microservices that handle authentication aspects, assigning all of them the “auth” tag would make it easy to find all authentication-related log entries. When a problem occurs within the authentication microservice, a team could quickly narrow down a large set of logs into those with the most relevance. Tagging logs with their source is a great way to maximize time spent looking through logs.
When issues or incidents occur with caching systems, it’s beneficial to have the ability to filter logs from all cache-related services quickly. Teams can problem-solve production issues without spending too much time filtering by context. Utility-based tagging is the most effective way to organize logs in Mezmo, and it’s configurable when ingesting logs into the system for the first time.
We can supercharge our development process by maintaining comprehensive and organized logs. Even so, the overwhelming amount of data taken in by Mezmo from all our organization’s systems can be daunting to comb through. Filtering and searching make debugging and resolution easier, and tags are one of the best tools at our disposal for empowering proper filtering.
Tagging logs from different sources with the purpose they serve for your organization is a quick and effective method for making logs more beneficial— for you and your fellow developers! If you’re ready to take control of your application and system logs, consider Signing up for a fully-featured 14-day free trial or joining the Mezmo community today.