Cost Advantages of a Cloud Log Management Solution

In the past few years, cloud based services have seen tremendous growth. This fast adoption can be attributed to their low cost of acquisition, ease of implementation and economies of scale. Everyday businesses and enterprises of all sizes are making the switch from on-site infrastructures to both hybrid and completely cloud-based log management solutions. Determining cloud logging cost advantages can be tricky at first. There is no one uniform way to determine how much you’re saving. An IT infrastructure is a complex beast with many different areas contributing to your total cost of ownership (TCO). But we do know that switching to a cloud based log management system will save you money in the long run. There’s a few different ways to determine your cost advantages. First, you need to evaluate what type of solution will best fit your current needs and also be flexible enough to keep up with your future growth. You’ll also need to know how to determine your unique TCO and how that affects your DevOps and engineering teams.

General Cost Advantages of Cloud Based Logging

For the most part, your monthly expenses from cloud logging is based off of log data volume and your unique retention rate. This holds true whether it’s LogDNA or another logging vendor. But there’s a few other things that can be overlooked that cloud logging also helps cut costs from. An integrated system gets rid of the need to have separate instances for security and access controls and will also seamlessly interact with popular DevOps tools like Pagerduty or JIRA. These are just a few examples, but even those can add up. Cloud logging systems can also scale to handle spikes in seasonal log variations. It creates a redundant system that always makes your logs available, even if your infrastructure is down – the time when you’ll need your logs the most!Additionally, cloud logging alleviates the need to rely on open source solutions that require you to hire engineering support to set up, manage and deploy systems. A cloud logging solution just requires a simple command line install. Index management, configuration and access control are just a click away in this case. This ease of use frees up your man hours for your DevOps team. With a log management system in place, your team can instead focus on core business operations. Cost advantages are difficult to quantify in this case, as they will vary from business to business and require an internal accounting system that takes into account inventory costs, engineer support salaries, and opportunity costs saved. But we still can look to the general cloud market to see how companies are utilizing cloud-based systems and cloud-logging platforms to cut costs.

Look to the Greater Cloud Market For Answers

Arriving at a clear TCO is a difficult number to compute. SaaS and on premise IT costs are not exactly the easiest things to compare. Internal operational costs with employees will always vary and fluctuate. Just because you’re applying some cloud services into the mix, doesn’t mean you’re going to be cutting off the entire on-premise IT staff. Their roles are changing and evolving by the minute – but they’re also not going anywhere anytime soon. We look to a compelling study conducted by Hurwitz & Associates comparing cloud based business applications to on-premise solutions. This white paper found that the overall cost advantages for cloud based solutions was significantly greater than on-site IT infrastructure.Here’s a look at some of the areas of cost that are averted when working with a cloud logging management system.

Cost Advantages in Focus

• Setting up IT infrastructure – which includes hardware, software and general ongoing maintenance accounts for around 10% of the total cost of setting up an on-premise solution.
• Subscription type fees, in which the majority of cloud logging providers offer is the main area of cost. Under these costs is the fact that you don’t have to create an underlying IT infrastructure – for example if you wanted run an ELK stack. That also means you cut down on personnel costs too.
• A pre-integrated system for both the front and back end functionality of your business reduces disparate integration complexity and lowers implementation costs.

These three examples are just a few of the reasons more businesses are shifting their focus to gain additional cost advantages.

Global Spending Shifted Toward the Cloud

According to the IDC, spending on cloud services is expected to hit $160 billion in 2018 with a 23% increase from 2017. Software as a service (SaaS) is the largest category, accounting for over two thirds of spending for the year. Following that is infrastructure as a service (IaaS). Resource management and cloud logging make up the greatest amount of spending of the SaaS spend this year. The United States accounts for the largest market share of cloud services – totalling over $97 billion, followed by the United Kingdom and Germany. Japan and China are the largest in Asia with roughly $10 billion combined. There is a wide range of industries that benefit from cloud logging, from professional services to banking to general applications. Many of these businesses would be better off streamlining and integrating a pre-existing cloud logging solution rather than creating their own or wasting precious resources hiring and maintaining an on-premise IT staff.

What Cloud Logging Helps Eliminate or Streamline

The first area to go is the operational costs in hiring additional engineers. Let’s use running an ELK stack as our prime example moving forward. Cloud logging platforms have cost advantages in three main areas: parsing, infrastructure and storage. First, it’s one thing to be able to grab logs and get them churning through the stack – it’s a different ballgame entirely to actually make meaning out of them. While trying to understand and analyze your data, you need to be able to structure it so that it can be read and make sense. Parsing and putting it into a visual medium allows you make actionable decisions on this ever-changing and flowing data. The ability to use Logstash to filter your logs in a coherent way unique to your business needs is no easy feat. It can be incredibly time consuming and require a lot of specialized billable hours. A quick Google search will show you the mass amounts of queries into creating just a Logstash timestamp – something that’s already ingrained and part of a cloud logging platform. Logs are also very dynamic. Which means that over time you’re going to be dealing with different formats and you’ll need to initiate periodic configuration adjustments. All of this means more time and money spent just getting your logs functional. You shouldn’t have to reinvent the wheel just to be able to read your logs. Next is just plain infrastructure. As your business grows -- which is what any viable business is hoping and striving for – more logs are going to be ingested into your stack. This means more hardware, more servers, network usage and of course storage. The overall amount of resources you need to employ to process this traffic will be continually increasing. An in-house log management solution consumes a lot of network bandwidth, storage and disc space. Not to mention, it most likely won’t be able handle large bursts of data when you have spikes in logs.When an error occurs in production is when you’ll need your precious logs parsed, ingested and ready for action in a moments notice. If your infrastructure isn’t up to snuff and falters – not only will you not be able to investigate your logs, you’ll also spend money fixing your failed underlying systems. Building out and maintaining this infrastructure can cost tens of thousands of dollars on an annual basis. Finally, all of your data has to go somewhere. You need to know where it goes and what to do with it. Indices can pile up and if they’re not taken care of, there’s a possibility that this will cause your ELK stack to crash and you’ll lose that precious data. A few things you’ll need to also learn how to do is remove old indices and have logs archived in their original format. All of this can be done with Amazon A3, but costs more time and money.

Flexible Storage & Pricing

In terms of storage, cloud logging ensures that you can store and have flexible data retention at a fraction of the cost it’d cost you to host it locally. Pricing is flexible and most important scalable. These two characteristics make cloud logging cost effective for any kind of business. LogDNA’s pay-per-GB pricing (similar that of AWS) is a good example of scalability. When you have an in-house solution, you need to increase your hardware every time your data increases. And being in the business of growth, predicting scalability is tough. A pay-as-you-grow pricing model allows you to bypass wasted cloud spend and only pay for what you need. Finding the perfect balance is more difficult the other way around. Overall, these many benefits and an overarching trend of companies shifting towards cloud logging solutions shows that there are multiple cost advantages with these solutions. Determining just how much you’ll save from a TCO standpoint depends on your unique situation and configuration -- just be sure to think through hiring, maintenance and hardware.