Most modern application and service logging functions track errors and events within a system. When an event occurs, the logging function automatically records it in the log file.
Logs can record all events, including an event’s metadata, in a system if you set them up correctly. Alongside the event type, logs usually record:
- Date of the event
- Time of the event
- Description of the event with an error code, if available
- Name of the virtual machine (VM), serverless function, or Kubernetes pod where the event occurred
- Name of the app or file causing the event
- Identification number of the event
Log files include information such as security status, an application's status, and a system’s or entire infrastructure’s general behavior. This information allows us to use logs in many different ways. For example, logs can help us monitor application performance or determine an error’s root cause.
Every developer has likely searched through log files at some point. There have been times when I felt like an archeologist digging through layer after layer of logs. It can be time-consuming, tedious, and headache-inducing. Worse, log files may be scattered across many places, especially as IT environments become more complex. Organizations increasingly deploy applications and infrastructure in public and hybrid cloud environments. The increased complexity in deployments creates a greater need to manage security and performance centrally.
We can review each application and device log file to monitor its performance and security status. But, it’s better to centralize logs in a single platform. Centralized logging reduces the risk of missing a log and helps to reveal patterns across applications and infrastructure.
Log aggregation is the practice of gathering log files into a central location. Here, a system can ingest the files, organize the data, and make it searchable. We can collect logs from:
- Applications
- Microservices
- Infrastructure
- Web servers
- Operating systems
- Security events
Log aggregation software tools, such as Mezmo, formerly known as LogDNA, bring together all of these different logs to help us search through the logs and analyze their data. Let’s look at the different log types we can aggregate into one accessible location.
Application Logs
Internally developed applications and services create application logs when certain types of events occur within an application. The logs could include warnings and errors for which developers have added logging calls in their code.
They could also consist of information, warnings, and errors logged by libraries and frameworks the application uses. Application log aggregation is essential for large organizations running dozens or hundreds of internally developed apps. Even small startups that are running multiple web applications and APIs benefit from log aggregation. If something goes wrong, all logs are in one place and easily searchable.
We can classify application logs according to event severity and response urgency. Having all logs in one place makes it easier to determine what type of response is needed.
Microservice Logs
Cloud services run on many machines, so service requests often include many service instances. Each service instance records information in a log file about what it’s doing. These are called microservice logs. These logs note errors, warnings, information, and debugging messages.
Microservice logs are a subset of application logs since they are essentially small applications. Log aggregation is beneficial for microservices because they often call on other microservices to complete a request. Without aggregated logging, it’s challenging to trace an error that starts in one microservice and propagates through others.
Infrastructure Logs
Infrastructure logs include logs from load balancers, object storage, virtual networks, and more. These logs can include custom logs — such as Linux Audit daemon (auditd) logs, syslog, and Windows event logs — from environments such as on-premises hosts and cloud providers.
Infrastructure logs can also include information about virtual network computers (VNCs) and virtual cloud networks (VCNs), and other resources, such as functions and API gateways. They can include logs from an entire fleet of hosts.
Web Server Logs
Web server logs contain information about web server events. Web servers like NGINX and Apache typically log every request they receive. So, aggregating all these logs in one location makes it easier to determine request volume and error rate in our entire infrastructure.
Operating System Logs
Server operating systems (OSs) typically log a great deal of data. For example, Linux and other Unix-like OSs keep detailed logs of how the entire system performs — both good and bad — using syslog.
If we’re running multiple VM servers, log aggregation helps you track the health of all of our servers. There’s no need to sign in to each one individually to read through its logs.
Security Logs
Log aggregation is essential for security teams. Security teams usually collect information from all important systems and security tools. Then they analyze the data to find suspicious or unusual events which might represent a security problem. It’s much more challenging to do this with security data spread across dozens of different systems.
Conclusion
We have discussed the types of information that logs capture, took a closer look at six different log types, and saw how to use their information. We also explored how log aggregation helps technical teams more easily find the information they need and trace errors across systems.
