LogDNA is now Mezmo but the product you know and love is here to stay.
In our ongoing efforts to provide our customers with a best-in-class log management solution, we’re excited to launch improvements to our Agent. This new, robust version (3.4) provides significant performance improvements for our customers. Specifically, we focused on two main areas of the Agent: error handling and endurance.
One area of focus for Agent 3.4 is our ability to better handle error states and ensure you are optimizing performance. Specifically, we wanted to eliminate noisy error logs as well as give customers more control over their retry files. With Agent 3.4 we’ve implemented:
We also focused our efforts on improving the Agent’s endurance. Specifically, we implemented:
The agent was enhanced to support tracking non-contiguous offsets, which allowed for concurrent uploads to be re-enabled without sacrificing stateful lookback.
As excited as we are for customers to try out this new Agent version and see these improvements for themselves, we know that we are only as good as the data we measure. Because of the focus on performance, we decided to share some of the data and metrics from our own internal testing. To help you understand the nature of this data, I have included a helpful glossary in the Appendix below.
The goal of benchmark testing is to simulate the Agent under expected conditions. We ran benchmark tests on an EC2 instance in the us-west-1 region with the following specifications:
Instance Type: t2.large
Architecture: x86_64
Processor: Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GH
Number of Cores: 2
Memory: 8 GB
Storage: EBS
And here were the results of the test:
Total Time (sec): 23
Total Lines: 4166932
Line Rate (lines/sec): 181170.95652173914
Total Size (bytes): 52245566.0
File Rate (bytes/sec): 2271546.347826087
Total Time (sec): 22
Total Size (bytes): 310600357.0
Total Number of Samples: 149
Average Request Duration (sec): 1.0604503484161074
Average Request Size (bytes): 2084566.1543624161
The goal here is to test the LogDNA Agent for an extended period under a pseudo-realistic workload. The following endurance test was run on an EC2 instance for 72 hours with the following specifications in the us-east-2 region:
Instance Type: m5d.large
Architecture: x86_64
Processor: Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GH
Number of Cores: 2
Memory: 8 GB
Storage: EBS
In these conditions, the logdna-agent consumed around 46 MB VmRSS, which measures how much RAM the process is actually using.
Outside of the initial program start (which is reflected in the spike in the graph below, and is expected), the Agent consumed less than 2% of CPU throughout the entire endurance test.
In summary, Agent 3.4 performs well under heavy load for a long period of time and in error states that require some form of recovery. Please check out our documentation for further details. And as always, feel free to reach out to me with any questions: albert.feng@logdna.com
Max Process Virtual Memory: the maximum process virtual memory observed throughout the benchmark process run.
Read More About: