You Got 99 Problems, But Logging Ain’t One

There’s never a dull moment in the world of running a startup. As founders, business and technical leaders, we are literally faced with 99 problems on a daily basis. Therefore, we truly believe that log management shouldn’t be one of them.

I would be deceiving you if I said we always wanted to build a log management system, and that our number one goal in life was to disrupt the logging ecosystem. But as Steve Jobs once said, “You can’t connect the dots looking forward: you can only connect them looking backwards.” And it all makes sense - all the dots aligned when looking back at our journey…

Y COMBINATOR 2015

We came up with LogDNA by accident; we built this system for internal use as we were frustrated with the competitors. We never intended our log system to be a standalone product but hey, things happen for a reason right?Let me rewind for a second. It was winter 2015, my brilliant co-founder Lee Liu and I were in Y Combinator’s Winter 2015 program. We had a company called PushMarket and we thought our personalization engine was going to revolutionize the eCommerce space. I’ll be the first to admit that we struggled to find "product market fit”, the holy grail for every aspiring startup. "Build something people want. Build something people want! Build something people want!!!" were the words slung at me at our weekly YC dinners. It was clear a few months after the program that we weren’t building something that people want, so we initiated a new strategy and a new game plan.We looked back at our PushMarket journey, we looked at every aspect of the product and the design, we worked harder to reinvent it. At one point in our journey to perfect PushMarket, we built our own logging system as a basic needs requirement. And as the old proverb goes, necessity is the mother of invention.

THE CURRENT LANDSCAPE IS INTOLERABLE

Looking at the logging space is very interesting. I’ll break down the basics for you:

1. The business model is “We want to help you succeed, but we will charge you for every gig that you store with us”.

We thought this model was rubbish. Instead of helping us succeed, we were limited. We restricted the amount of log data we sent for fear of being overcharged. We would constantly change logging code to log less or log more compactly. This was not a good use of our engineering resources. So we ended up with a goal to design a system to give developers the flexibility to log everything, we wanted to take the away the unnecessary requirement of creating and maintaining a ELK stack, and we wanted to alleviate the stress of log management altogether.

2. The industry limits the search window for logs.

The standard “2 weeks” retention window was too brief. There were many occasions where we needed to search 4 weeks, 8 weeks and 6 months back. We desired to alter this.

3. User interface and user experience is clunky at best.

They gave us a “take it or leave it” option. We needed to change the choices. We wanted beautiful UI, but more importantly, awesome UX so that you can jump in, get what you need and go back to working on your product. The word frictionless comes to mind and is one of our design goals.

OUR “SLACK” MOMENT

I attended a dinner late last summer when a few tech friends of mine were criticizing and grumbling about their log management system. It shocked me that their frustrations mirrored ours; we were not alone in the dissatisfaction. A few weeks later I started to ask others. The general consensus was that the current systems are unsatisfying in terms usability and functionality, but the major concern and aggravation was storage issues and the expense related to data storage. It’s 2016! That’s the equivalent of having a monthly mobile phone plan and still being charged by the minute!It was clear that we were onto something, that’s when we had our “Slack” moment. We realized that we built something internally that was more powerful than our original business plan. Similar to how Slack started off as a MMO gaming company before they became the illustrustrious, enterprise company they’re known for today. Their transformation was ingenious, and we were inspired!And just like that, LogDNA was born.

THE JOURNEY BEGINS

I don’t know what the future holds, but I’m glad we are going down this path. I encourage you to give LogDNA test drive! I’d love to hear your feedback and suggestions on how we can make this space better.We’re already planning to leverage infrastructure data and apply machine learning to help forecast certain future outcomes. We’re developers ourselves, and it’s gratifying to have built something that every team needs.Here’s to a great 2016!