Why the Mezmo Agent Runs As Root

One question that customers often ask is “why does the Mezmo agent need to run as root?” With IT departments and DevSecOps teams pushing to secure systems against cyberattacks, running a cloud-based logging agent as root sounds like a huge risk. While it’s true that you should avoid running applications as root, there are several reasons why our agent runs as root out of the box and several ways that we reduce your risk of attack.

Getting to the Root of the Problem

On Linux, Unix, and Mac computers, the root user (also called the superuser) is a special account with full access to files, applications, devices, and other resources. Root can perform almost any action on the system without restriction and regardless of any access controls. Applications and processes can also run as root, which grants them the same level of unrestricted access. Because of the security risk, the root account is usually either disabled or password protected by default.

Some processes—such as the init process—can only run as root since they need a significant amount of system access. These processes often create log files containing important diagnostic information about the system. The problem is that the log files are created under the root user account, making them inaccessible to other users.

The Rationale for Root

In order to send these logs to Mezmo, the agent must be able to read these files. The challenge is doing so in a way that doesn’t modify the original file, is easy to deploy, and doesn’t expose the system to a privilege escalation attack.

Other logging and monitoring applications have tried using methods that don’t require root. One approach is to use a separate non-root user account or group. For example, the adm group is meant for system monitoring and has access to some system log files. However, the only way to monitor all log files using this method is to add your account to every group that owns a log file. Since some logs are owned by the root user and group, you would still need root-level privileges to access these files anyway.

Another approach is to modify your file permissions to allow non-root users using chmod. The problem with this approach is that it modifies the file’s metadata and exposes its contents to underprivileged users. It can also cause problems with other programs that access logs, such as logrotate.

Mezmo takes the approach of running as root for three main reasons:

1. It uses kernel-level file monitoring to detect new lines in log files.
2. It monitors restricted  paths such as /var/log and /var/data.
3. It monitors logs from processes that run as root, such as Kubernetes.

Other logging agents and log collectors also use the root approach. On Kubernetes, Fluentd and Filebeat require root in order to mount the host’s /var/log directory. The official Kubernetes Logging Architecture documentation recommends using root-based solutions for logging clusters on the node-level. Some logging solutions use syslog instead of an agent, but even these take advantage of the fact that syslog itself runs as root.

Staying Protected

Running any process as root has risks, but we take several steps to help you stay protected.

First, all data sent between the agent and ingestion servers is encrypted. The agent uses HTTPS by default to ensure your logs can’t be read by anyone else. We’re also fully compliant HIPAA, SOC2, HITECH, PCI-DSS, GDPR, and Privacy Shield.

Second, the agent doesn’t need special permissions when running on Docker or Kubernetes. Other agents require privileged mode, which gives the container host-like capabilities including access to the network stack and devices. The LogDNA agent container only needs to mount your host’s log folders, and only while the container is running.

Third, the agent is completely open source. You can clone or fork the repository, audit our source code, add your own changes, or build your own binaries. You can also submit a pull request or open an issue if you find a bug, have a question, or recommend a change.

In addition, here are several additional steps you can take to secure your systems:

• Follow the principle of least privilege: only grant processes the fewest permissions needed to complete their tasks.
• Configure your firewall to only allow traffic to and from api.logdna.com.
• Don’t run untrusted software and review the source code whenever possible.

If you have any questions about security and Mezmo, please contact us.