Debug Logs and Analyze Trends with Log Data Rehydration

Everyone in your organization needs logs to perform the critical functions of their job. Developers need them to debug their applications, security engineers need them to respond to incidents, and support engineers need them to help customers troubleshoot issues. These various use cases create general requirements for enriched log data, often including accessing insights from outside typical retention windows.

Having so many use cases creates complexity, making everyone's jobs more difficult across many teams.

At Mezmo, we introduced log data rehydration (a.k.a. log restoration) to enable use cases involving data access beyond typical retention times without adversely affecting costs. Log data rehydration allows users to store their older log data in cold storage, away from Mezmo, and then recall it later if or when needed. If you'd like a deeper refresher on log data rehydration, check out this blog, where we introduced the then-new feature.

In this blog, I’d like to examine some real-life cases where log data rehydration might be necessary.

Debugging Logs for Extra Credit

I've got news for you; your system's got bugs. Nobody makes a perfect build; you're a human, and humans are beautifully imperfect. If you're one of the lucky humans, you may find yourself with some extra time on your hands to fix some of the tiny (or more significant) imperfections in your systems. As stated before, you must use log data to debug any system. Still, if the last incident occurred outside your retention window, you'd ordinarily have difficulty fixing your issue.

Without some rehydration functionality, you'd probably have to increase your retention window to a time that may not be cost-effective to store your logs in hot storage. Even then, if you were to increase your retention window after your bug occurred, there’s probably no guarantee that you’ll get the log data you need to fix your issue. You could also manually comb through files of old and raw log data that you may have stored on a hard drive somewhere, but that would be tedious, time-consuming, and may not get you the analytics you’re hoping to find.

Suppose you utilize Mezmo to rehydrate your log data. In that case, you could recall that older log data containing the bug you're trying to fix back into Mezmo’s UI to perform an analysis and ultimately fix your bug. Recalling old log data back into the Mezmo UI doesn’t just make it simpler and easier to digest for human consumption; it allows you to perform analysis on those logs to unlock the power of your data.

Root-Cause Analysis

Suppose everything has been relatively copacetic lately, but you’ve noticed an interesting trend in your log data, and you’d like to figure out the root of your new trend to prevent things from getting out of hand. To perform a root-cause analysis, you could utilize log data rehydration from Mezmo by storing your older log data inside an S3 bucket and then recalling that data back into Mezmo as needed.

All you need to do is define a time and date range inside the Mezmo UI under the "Manage Restoration" page. Once you rehydrate your log data into the Mezmo UI, your logs will appear as they were when they were new, and you can perform any analysis that you would typically be able to inside Mezmo.

It is essential to mention that doing this doesn’t require any specialized knowledge of AWS; as long as you can stand up an S3 bucket and establish a connection to Mezmo, you can recall your log data back into the Mezmo platform from directly inside the UI. The same applies to other types of object storage as well.

Conclusion

Mezmo enables a myriad of other use cases with log data rehydration. Chances are, there are even some that we're not considering. Tell us how you would do it!

Take our free trial and test out our log data rehydration capabilities.