DevOps State of Mind Podcast Episode 6: The Future of DevSecOps with EMA
LogDNA is now Mezmo but the DevOps State of Mind podcast you know and love is here to stay.
Chris Steffen is a research director for information security at Enterprise Management Associates. EMA is a leading analyst and consulting firm that prides itself on going beyond the surface to provide deep insights about the IT industry.
I'm Liesse from LogDNA. Before we dive in, I just wanted to take a moment to thank all of you for tuning in to season one of DevOps State of Mind. We took a break over the holidays and are back with more awesome guests to talk about DevOps, security, and the future of tech. For the first episode of the year, Chris is sharing his DevSecOps predictions for 2022 and chatting about what it takes to compete as a security vendor today. Join us as we get into a DevOps state of mind.
Liesse Jones: All right, Chris, thank you so much for being on the podcast.
Chris Steffen: Absolutely. Thank you for having me.
Liesse Jones: Yeah, this'll be a fun conversation. Before we get into it, I would love for you to just give a quick intro of yourself for the listeners.
Chris Steffen: Yeah, sure. My name's Chris Steffen. I am a long time security practitioner and a technical evangelist. I’ve worked with a number of different companies in that role throughout the last 20/25 years now. I have always focused on security, be it in the cloud or from a regulatory and due diligence perspective, but have lots of experience down there. All the certifications behind the name and so on and so forth. Right now, I am the lead research director for information security at EMA, where I get to talk with folks like you and conduct research on emerging trends in cybersecurity and the realms of risk and compliance.
Liesse Jones: I love it. I'm so excited to talk about the future and what people should be focusing on this year. In the last, you know, 12 to 18 months, there have been no shortage of risks that have come to people's attention. And some major ones that have impacted multiple industries at once. So let's start there. What do you think the number one risk is that industries aren't paying enough attention to?
Chris Steffen: The one that I'm going to be talking about in 2022, and I've already talked about it a little bit in 2021, is the risk of the developer. I know that's part of the reason that we wanted to talk today and I think it's worth really diving into.
In the past you've always seen information security programs focused on networks. And then you had a lot of identity security, and obviously everybody's talking about ransomware and I'm not trying to take away any of those things. And those are all things that are very important and will continue being important.
I think that you're going to start seeing a real focus on the bad guys going after developers. Developers tend to be very well credentialed individuals within the organization. They have access to obviously tons of intellectual property—the keys to the kingdom as it were. And, the bad guys who are also developers obviously understand that. And instead of going after some admin assistant or bookkeeper or something like that, which may be an interesting target too, they've decided to go over and start really focusing their attacks on those that they know, because they are those people that have access to key systems and key information within those organizations, and attack them directly. With the hopes of hitting the jackpot a lot sooner than they would with a security administrator or something like that.
Liesse Jones: Yeah, definitely. I would love for you to talk a little bit about the SolarWinds attack. You've told me that that was really a developer focused attack.
Chris Steffen: The SolarWinds attack specifically—you know, it depends on who you believe and what you read, but everything that the security realm understands is that this initially started out as a hunt for shared secrets. Shared secrets is one of those things that is the dirty little secret in every development house, in every organization, that likely somewhere in the code that you have today are logins and passwords that are highly credentialed and have access to every part of the organization.
Again, developers know this because they use those to take and log into APIs. They use those to log into networks. And the bad guys are going to start focusing on those directly. And so what you're going to see from the SolarWinds attack specifically was a search of code to try to find some of those shared secrets, which then evolved into the massive breach that it ended up being.
Liesse Jones: Yeah, you just mentioned APIs. Can you dig into that a little bit more and why you think API security is so important specifically?
Chris Steffen: Yeah, absolutely. So, when you start talking about the glue that holds the applications together, APIs are it. In fact most of the time you won't even consider taking and buying a third party solution or application that doesn't have those API integrations already in the box or as part of the development solution that's going to happen when the product is purchased. Again, the other dirty little secret is that so many of these APIs are either rushed out the door or haven't been updated or have a certain amount of vulnerabilities in them that are really left unchecked and untested when you're doing code tests or examinations or what have you, you take them at face value as being secure.
So let me give you a very simple example. Again, I'm not picking on anybody in particular, but most organizations that do business with a third party, in some kind of methodology with the code are going to be required to either have an open API, which is an open standard that they're going to use to connect internally to systems, or a custom API. And that custom API being something that they create with or without support from the other company and to connect specifically to their systems. Once that API is created—and this is just the human nature of how things work—you rush to get it out the door because it’s imperative to get it out the door as quickly as possible to get business up and running. There's a lot of pressure from the business side to get that API up and running. But once it's out the door, you never ever want to screw with it again. It is static. You don't want to break something that's working and yes, there might be updates or reasons that you need to update it. And maybe you've looked at it. But a very large percentage of the APIs once they are created are never messed with ever again.
Well, let's take an example of a company that created an API for a solution in the early two thousands. Let's say 2006, 2007, 2008. It stands to reason that that API could still exist or at least the foundations of that API could still exist. And it could be still using the depreciated securing algorithms like TripleDES or, or something like that, instead of something more robust. And again, that sounds convoluted but that's an example that I know exists today. And so that is just one avenue where APIs pose a real risk to the enterprise. Once APIs are created, there really is no incentive for a developer organization or the business as a whole to go in and constantly mess with those APIs. Again, just human nature not to screw with something that's working.
Liesse Jones: That's super interesting. Is there a way that people can test those APIs before bringing on a vendor? Or any way for them to be a little bit more proactive about the security?
Chris Steffen: Yeah. I mean, there's lots of things. You can actually have code scans run and find out if there are vulnerabilities known, find out if they are using open libraries that have been depreciated or what have you. So there are things that they can do. One of the things that I always recommend and it's becoming much more prevalent is that those companies that are in the process of going through mergers and acquisitions, they should always be taking and engaging in a security firm to take and do some real due diligence of the company that they're purchasing and make certain that those kinds of vulnerabilities are at least known. You don't even necessarily need to immediately solve everything overnight, but you’ve got to know that you have a problem before you can fix having a problem.
Liesse Jones: Absolutely. Outside of the API stuff, what would you say are the most important tools or practices that companies should be thinking about for protecting shared secrets?
Chris Steffen: The one that [I] and every security person in the world will tell you, is following best practices is the best place to start. Right? And what I mean by that is that, you know, not taking and having shared secrets, having company policies that don't allow you to have shared secrets, doing code reviews and independent code reviews to make certain that the code is secure, that the code is passing some of those basic requirements. Those very simple best practice ideas will always get you 95% of the way there. And that's 95% better than a lot of companies are. So start there. Once you get past that part and you still want to go to the next step, then you can take time to really get into third party code scannings and quarterly, or some kind of timely, independent review of your entire code base.
I would always recommend that your documentation be updated at the same time. I know developers hate documenting—again, I’ve been there, done that—but it's one of those things where it really is a best practice. It really is critical. And I know that it takes time. I know that it stops the bus from getting code out the door. It's really incumbent upon the business and the business leaders to make time for developers to do some of those best practices. I know that there is a lot of pressure on developers and development leaders to get code out the door because resources are tight and blah, blah, blah. Everybody understands all that. But I don't think that we really fully engage at the executive level, at the C table level, the level of effort that it takes to really do some of these best practices right. The right regression testing, the right kind of code analysis, the right kind of security testing, the code review and the documentation and such. And again, I know it takes time. I know nobody likes to do it. And I know that you don't like to be held accountable for stuff that you're being told to do, but then conflicts with being told to get it out the door and blah, blah, blah. Again, I get all that stuff. I wasn't born with the last drop of rain, but it really starts with a tone from the top for those business leaders to understand that as well. And it really comes down to communication. I don't think that business leaders really understand how development and developers actually work and function within their own businesses. And so helping them understand that is always going to be an important part of the process as well.
Liesse Jones: Have you seen that change as some of these bigger security issues have come up? You know, every executive gets scared when they see a major breach that impacts them and a ton of other people in the Fortune 500. Has that changed the perception?
Chris Steffen: It has and it hasn't. I don't think there's any question that we are more security minded than we were, let's say even five years ago. And I applaud that, but we also have a squirrel mentality. Right. So it's always the next shiny object. Right. It's always the next thing and what's driving business and are you making me money today? “Thank you for what you did for me yesterday. What have you done for me today?” Kind of attitude, right? I wish it wasn't that way, but that's just reality. And so, I do think that there is a greater focus on the security aspects of the business and how important that they are. But, given some of the other requirements that business has and some of the resource constraints that they have, I know that sometimes they take a back seat. Sometimes there's going to be an event that puts them back to top of mind again, but then it's just a matter of time before they take the backseat again and so on and so forth.
The other thing that's really kind of changed that perspective is the sheer number of regulatory and compliance related regulations that have taken over business. It used to be that PCI was the quote unquote “gold standard,” which again, if you're a security person, you know just how hilarious that is. Now, almost every vertical has their own set of standards that they have to comply with. And so whether you map them to security controls or not, or however it is that you decide to do them, security and compliance generally go hand in hand and those companies still have to pay a certain amount of attention to those standards in order to be in business. The old joke is that companies want to be more secure, companies have to be compliant in order to be in business.
Liesse Jones: Yeah, absolutely. I think you also have the benefit of consulting with a ton of companies so you get kind of an insider perspective into the questions that they're asking.
I can imagine, you know, sitting on a call with a client and them saying, “Okay, these are three things you're saying are important. What would be the number one thing that I actually have to do?”
Chris Steffen: Pick one. Yeah, right. I think that we live in a world where—again, I'll just give you a simple example.
Let’s take a regular run of the mill, chief executive officer. And they are tasked by a board of directors to improve their security because they saw something on the nine o'clock news that freaked them out. And the border of directors says, “what are you doing today to combat this?” And the CEO says, “I don't know, I will get back to you and we will do something.” And so they go to a CIO or CISO and basically say, “Go do something. Here’s some money to spend to do it.” And so they do, right. And the CEO goes to bed at night thinking, “well, we spent money on something security yesterday. So we're now secure.” And even the CIO may say, “well, sure, we spent money on a security product yesterday. Therefore we must be secure.” Whereas the part that we sometimes forget or don't acknowledge is that there's 500 gazillion aspects to security and we can't possibly cover them all with the, you know, $10 budget that you got out of the bubble gum machine.
So, it's important to realize that it's an ongoing and continuous process and it is part of a larger overall strategy, not a one and done kind of thing.
Liesse Jones: Yeah, absolutely. If there was one or two things that you could say companies are really focusing on this aspect right now, or they're planning to focus on this in 2022, what would that be?
Chris Steffen: Well, I know one of them that is coming up more and more is—and it has for a couple of years now, to be honest with you—is how to deal with data and data privacy sort of stuff is critical. And so let's unwrap that for a second. There are plenty of regulations going around depending on what you believe in from GDPR to the CCPA out of California, to the new PIPL in China, and depending on what markets you're in or what markets you believe in, or you want to be regulated by depends on what regulations that you're going to comply with. But for the most part people are pretty aware of GDPR and CCPA. There isn't much to be said about the Chinese one yet because it's literally brand new. So the first thing that you have to do in order to be able to comply with those regulations is understand what your data estate really looks like. That is a huge task. That is not something that's very simple to do. For example, in any given situation an organization may have—I'll just be very conservative and say a dozen different spots that data might be stored or located or parsed. And that could be cloud locations, that could be data stores that could be databases so on and so forth. What information you have on Chris Steffen could be located in seven of those 12 different data stores and combining them all and reconciling them all to know what data is stored, where, on Chris Steffen is a huge task because you can't possibly begin to comply with some of these regulations until you can emphatically say that I know where every single piece of information on Chris Steffen is located in my company, because if you miss one, you're not compliant. And so that is a huge undertaking for many, many organizations trying to get a handle on the data estates and what that really means to their organization in general. So you're going to see compliance organizations, you're going to see security organizations, and all the infrastructure around those spending a lot of time and effort over the next few years, trying to reconcile how to deal with those kinds of regulations.
Liesse Jones: Yeah, that's great. Earlier in our conversation, you also mentioned M&A and things that people should be aware of when they're looking to acquire a company. I'll ask you on the flip side, what do companies that are hoping to be acquired in the next year or two need to be doing right now to prepare so that they're in a good position when that time comes for them?
Chris Steffen: Great question. I think the first one and the most fundamental one always is to have a firm understanding of what your value is. Is your value in people? Is your value in code? Is your value in data? Invariably, it's going to probably be one of those. So when you realize that it's one of those things, then have an understanding of how to protect that value and how to ensure that value gets transferred to a potential buyer.
What do I mean by that? So if you are, let's take the simple one of an application or code. If you are writing code, that code should be solid. That code should be without flaws. That code should be easily distributed and updated. It should be well documented—the documentation monster comes again. It should be well understood what are the inputs, outputs, nuances, and dependencies of that particular code. Because the first question that I'm going to ask as an M&A advisor are those. Do you have relationships to make certain that this code is maintained [and] is secured? What is the longevity of this code? How's it updated? So on and so forth. And that's just a very simple example, but it's an example that really ends up taking and putting in roadblocks for companies looking to sell a code-based idea, an app, something along those lines. Because if you don't know those answers, if you can't go to the person doing the M&A evaluation and say, “here is why it's valuable, here's how we are protecting this asset for you” then I have real questions about the long-term stability of buying that particular software, service, whatever have you. Especially given the—and everybody knows this too—the turnover that we're seeing in the industry from talent and work, having that kind of certainty when it comes to that particular asset is critically important. If you don't have that kind of surety, then your value is less because I can't tell you what that code is going to look like 24 hours after the acquisition has occurred. And if I can't do that, then the value of that asset is lost.
Liesse Jones: That's a great answer.
And I love that you talked about the turnover of people as well. That's no surprise, all of us are really feeling it right now with the great resignation. And I think the other side of that is that the market is really saturated with security adjacent tools. And so not only do you have people who are maybe moving from one company to another, where their skills might be directly applicable. But then you also layer on the fact that you're just competing with a ton of different vendors to provide solutions for all of the things that we've talked about today. And so I would love to know from your perspective, what you think it takes to compete as a security vendor who's trying to solve some of these problems that you're talking about.
Chris Steffen: That's a great question. And the reality of it is that it's a huge answer too, right? It isn't any one thing. I think the things that are most top of mind for executives looking to purchase a tool today is ease of use and immediate return on investment. So what I mean by that is that, how complicated is it and is it going to take my security squad the better part of a year to implement? If that's the case, then I'm not interested. You figure that the lifespan of a technology executive is something between 24 and 30 months as it stands right now. And if your implementation, if your solution, takes longer than that, you figure that the same champion that you had to buy the product is not the same champion that you're going to have to implement the product and run the product.
And as you well know, as a security vendor, that is a huge problem because you'll get different ideas in there, people want different things, different priorities. And now all of a sudden, as a security vendor, you are the last man standing, because the person who wanted the solution, the person that championed your solution is now the person that is working over at ABC competitor. And so that is a huge, huge problem.
The second part that I would say along the same lines is be very honest about the level of effort that it takes in order to really start seeing results with your product. And I'm not talking about the initial install, I'm not talking about some vaporware nonsense that nobody really cares about. I'm talking about when I buy your solution I will start to expect to see real results after initial configuration and what have you, in X amount of time. If you can't provide a real estimate and a real example of that amount of time, then you don't have the right product. And it's going to be very, very hard for you to compete in a market where that kind of expectation is becoming the table stakes when it's all said and done.
Liesse Jones: That's a great answer. And I think the first part of it really resonated. I've seen in my career so many times where a new executive comes in and it's twofold, right? They don't want to take responsibility for a tool or process that the person before them implemented because they don't know if it's going to work and they don't want their name attached to something. And also they want to make a strong impact in their first 90 days or however long it will be. So I can imagine from a security perspective, that's really tricky unless you're bringing on a vendor that you know is going to be able to deliver in a certain amount of time.
Chris Steffen: That's exactly right. And, you know, one of the other things that I would mention too, is be very aware of the level of effort that it takes after the purchase too. It's one thing to have a great product. It's one thing to have support getting it set up. But if it takes a PhD in order to run it because it requires a whole different, you know, markup language or admin language or a set of tools or certifications that are not exactly commonplace. Again, it isn't like those people are a dime a dozen on the best days and the way the market is right now they may not even exist at all. And so you could be buying a tool that the second that person that sets it up leaves is an absolute boat anchor that is sitting there and depreciating and your CFO asks you daily, “are we getting our depreciated value out of this particular service or application?” when you're literally not even using any of the function of it, but you're still paying for it because the guy that you had went to go make 20% more working out of his local Starbucks and doesn't want to be in the office every day. And therefore you are left with a solution that you literally don't have the ability to use.
Liesse Jones: Do you think that automation solves for some of that, or does it actually introduce a new risk by people being too dependent on thinking the tool’s kind of taking care of itself?
Chris Steffen: Yeah, I think both, right. I think that there are great parts of automation that solve problems that were basically consuming resources that now you don't have to have a warm body taking and doing that particular task anymore.
Take something very basic, like firewall log reviews and those kinds of things. Speaking as somebody who's had that job, it sucks. It's the worst. And so nobody wants to sit there and be the log review monkey. It's not good. You have automation that takes care of a lot of that, but that person now can be utilized doing architecture and setup and some of the compliance related things that we were talking about before. You hire somebody with 37 technical certifications, when it's all said and done, quite bluntly you don't want them to be the log monkey. You want them taking and doing architecture. You want them doing, for lack of a better term, big brain kind of things, because you hired their big brain, you did not hire them to do something that you can do from an automation perspective.
The other side of that is that automation takes and brings a certain level of complacency that we believe that this is all being taken care of, but if it's not configured correctly or not maintained correctly or not monitored correctly, it doesn't matter how good the automation is. Automation of junk is still junk, right? Garbage in, garbage out. This is the old developer credo when it's all said and done, but that's really the case with automation as well. If you don't have it configured correctly, it doesn't matter how good the automation is. You're still monitoring junk. And so monitoring junk doesn’t solve any problems for you? It just, in fact, arguably creates more problems because now you're not only not monitoring the stuff that you need to, but you're monitoring stuff and paying attention to stuff that doesn't have any relevance to where your actual security problems might be or where your risk for the company might have.
Liesse Jones: Yeah, I think that that's a great thing to bring up. And it's really interesting because on the next episode of the podcast, I'm talking to Nick Durkin, who's the field CTO at Harness and their whole thing is removing the most annoying parts of your job so that you can focus on what you do best and they're using AI and ML to do that. So it kind of plays off of what you were just saying. You know, you hired these people for their big brains. Let them actually use and grow that muscle.
Chris Steffen: One of the things, if you remember—now I'm dating myself here—there was the virtualization wars between Microsoft and VMware. And one of the problems early on was people constantly talking about virtualizing themselves out of a job. We all know, you know, 10 years, 12 years, 15 years later, that that was nonsense. That was never going to be. And as one of the the companies that really pioneered some of the virtualization processes and stuff, working with Microsoft, I can tell you immediately that we were able to take and focus that the MCSEs that we had on staff at time and focused them specifically on architecture and testing and development and those things that we really wanted people with those kinds of skills to do.
Whereas all the administrative stuff that you had server admins doing before really just became the function of a more junior administrator. And so not only do you end up taking and saving money that way, but you end up with happier, big brain types and happier big brain types right now is a really important thing.
Liesse Jones: Yeah, absolutely. Okay, this has been an amazing conversation.
Chris Steffen: I’m not ready to be done!
Liesse Jones: I'll have to have you back on!
Just to recap for everybody, some of the things that they should be focusing on in the new year are API security, protecting their developers because they are going to be the target of choice, and focusing on DevOps and compliance risk.
And just thinking about what it takes to compete is focusing on the people, using their brains to really innovate and create solutions that are easy to use, provide immediate value, and where you can really show the time to value for your customers.
Chris Steffen: Yeah, that's absolutely right. And I always finish with, always remember the basics. Always think about the basics of basic patch management, understanding where your risks are, and understanding what the priorities of the company are. Those things alone will usually take and help set what the security agenda needs to be and what the risk profile for the company is going to be that the security team needs to focus on.
Liesse Jones: Amazing. Chris Steffan from EMA everybody. Thanks so much for being here, Chris.
Chris Steffen: Absolutely. It was great. Thank you.