Dr. Eric Cole: 5 Security Metrics To Track Your Cyber Security Success

Excellent. So if we look at what's happening at cybersecurity, it seems that organizations are spending more money. They're hiring more people, but they're not necessarily addressing the problem. The problem seems to be getting worse and not better.

Now, there are some people out there that wanna give undue credit to the adversary and they want to go in and claim that the adversary is super advanced and it's this nation, state attack. And certain really large companies wanna make these ridiculous claims that while they don't know how they got compromised, they're confident that it required a thousand people in order to break in.

And to me, the problem is we're not accepting the responsibility that we need because we don't have the proper visibility with metrics. And instead of fixing and improving our security, we're blaming saying it's advanced nation states. There's nothing we can do about it. And I think that is false.

Yes, there are advanced attacks, but let's face it. Whether we're looking at recent breaches like solar winds or others, those attacks we're preventable, and those attacks were detectable. The problem is proper metrics were not in place to see how well their security is performing.

And, and just to show you the importance of metrics, I want to take you on a little journey of it because information technology as a discipline is very mature, very robust, and there's a lot of lessons we can learn from it and cybersecurity because they're about 15 years ahead of us. So let's go on a quick journey. Starting in the late eighties, organizations started buying computers. I remember when I worked at a defense contractor in the eighties, we had the old Wang Tempest computer systems that were in play.

And if any of you worked on them, you, you had the forearms really strong because they had eight rows of nine screws each. So you actually had to do 72 screws to open up those systems. So you had really good forums doing that. And they started using 'em for basic applications. So as computers were introduced, companies realized they were important to an organization. So what did they do? They created a manager of information technology, and because it wasn't that big in the eighties, they buried it under operations.

And then as we moved through the nineties, 91, 92, 93, boom, it seemed like overnight. All of a sudden organizations realize they are depending on technology and computers for their lifeblood. So around 94, 95 organizations did two things.

First, they realized that burying it under operations, wasn't good. They needed to have direct access to the executive team. So many organizations starting around mid nineties started creating chief information officers that reported directly to the executives. So now they had firsthand knowledge and information on what was happening while that was important. That wasn't the critical piece. The critical piece of the equation was having a clear metric so everyone can be on the same page.

So if you've ever worked in it have been a CIO, you know that the common industry standard is five, nine S 99.99, 9% uptime availability. And what's great about that is everyone's on the same page. If that CIO delivers 99.9, nine, 9% uptime availability, the executives are happy. The company is happy and the technical team is happy. Everyone recognizes that their focus and mission is to achieve uptime availability. And that's the primary driver. And everyone is on the same page. That is a key component because having clear metrics allows you to measure and see how effective you're doing.

That's why it is so mature. Now let's rewind. Cybersecurity. Really didn't start getting kicked off until the mid to late nineties. Yes. In the early nineties with email, you had the, I love you virus and the Melissa virus, but, but even then it was still in its infancy. And companies had a security officer, but they were buried under it.

They, they weren't have visibility or exposure. And then over the last 10 or 15 years, starting about five years ago, organizations recognized you can't bury security under it. It is the conflict of interest.

Now, any CIOs on here that are old school, you might get mad at me, but I'll give you the data. And I'll stick by that security under it is a conflict of interest and it's simple. Security has three things, confidentiality, integrity, availability it, and CIOs have one thing availability. So if the CIO is bonus on availability only, and that's what they use to determine whether they keep their job and you now add confidentiality and integrity, it is not going to get the proper focus that it's needs.

And now with all these breaches that have been happening, the good news is most organizations are taking the CISO, the chief information security officer, and putting them equal to the CIO. So that problem is solved or being solved as we speak. The problem is the metric. The current metric that's used by organizations. The five nines of security that organizations are using today is a flawed metric. And that's the problem. What is the metric? The five nines of security that almost every organization uses today.

If we don't have a breach securities doing their job, and you might say, Eric, how do you know that? It's simple. What happens when a breach occurs? When a breach happens, the CISO gets fired, right? That's pretty much playbook of cybersecurity. And the reason is because that's the metric the executives are using. The executives are saying our metric is we should never have an attack. And if we have an attack, that means security failed. And we fire the system.

Now, unless you like switching jobs every nine to 12 months, right? That's not a good metric, right? That's not an effective metric because breaches are going to happen. And if you are judged on something that's inevitable, then that's going to not work out too well for you. So I've been studying the five nines of security for five years, and I will tell you the metric, I'm going to give you today following by five sub metrics.

I'm a, I am gonna deliver on what my title is. Isn't perfect. It isn't a perfect metric. And I wish I had a better metric, but here's what I found. After five years of analysis, I can't find anything better. So we are using this with our clients. So I just want a caveat cuz when I give this presentation and I give the five nines of security, security, people love to criticize, right?

That's, that's what we do so well, we find fault in issues. So I have people going, Eric, I, I don't really like your metric.

I'm like, great. What's a better metric. They're like, well we don't have a better metric. And I'm like, well it's my metric. Better than the current metric. If you have a breach, you deliver security.

They say, yes. I'm like, then use it, right? Stop fighting the inevitable. We're not looking for perfection here. We're looking for something better than what we have. And right now this is the best that we have. So the current metric that I've used with my clients, we've used over 77 times.

It's very, very effective metric. And my response to you, if you don't like it, use it and see how it works out for you. But here's the metric attempted attacks. What are the number of attempted attacks against your organization?

Now, first, this is a lot better than saying we're not gonna have any breaches. And if a breach occurred, you failed. So it's accepting the reality. So right outta the gate attempted attacks. My metric is a hundred times better than the current metric that's being used today. Second reason, I like this metric. It's a positive metric. It shows what you're doing. One of the metrics that people like using, which is not a good metric is vulnerability management data. The problem with vulnerability management data is it's a negative metric.

So you go to the executives and you say this quarter, we had 300 vulnerabilities. You that's negative. You're telling them what you didn't do. You're basically saying there are 300 things on the network that we did not address correctly in our environment. That is not a good metric. This metric is positive. It's telling you what you are doing, right? It is going in and telling you what you're doing correctly. So that's the second thing. And then the third thing is it raises visibility.

I will tell you right now, most executives I work with and ask your execs, go into them and ask anyone of your executive, how many attempted attacks do you think we have on a daily basis? And the average answer I get when I work with CEOs is three to five. Now anyone that's worked in this field know that it's more like 15 to 20,000 a day depending on your organization. So now attempted attacks give you that positive metric. It raises the visibility and it now addresses that real problem.

So what I like to do is I finish up the way I always give my presentations is I give you what you want, which are the five metrics. But I tell you what you really need to know, which is the five, nine metric. So what you really need to know and put in place is the five, nine S of security, which is attempted attacks need to replace the current metrics you're using today. Other five metrics that you can utilize, one is average time to detect and respond because remember prevention is ideal, but detection is a must. So I'm gonna measure the maturity of your organization.

Can you tell me how long it takes for you to detect and respond to an attack? Now, careful here, if you come back and go, Eric, I absolutely can. We haven't been attacked or breached in over two years. So it it's zero.

We, we catch and prevent all attacks. Okay? With love in my heart. When I say that, that means I'm, I'm gonna, I'm gonna be a little harsh here. That means you failed, cuz let's face it. If every organization on the planet is getting breached and compromised and you are telling me that you didn't have a single attack or breach in two years, that either means you are not looking in the right place or you have magical unicorn grade security. And trust me, it's not the latter. So whenever I work with organizations and I go, or we're really willing, mature, we prevent all attacks.

We haven't had any attacks in two or three years. It's really a sign that nobody wants to admit that you're very immature because you cannot prevent all attacks and attacks will happen and you need to change your focus. So mature metric, average time to detect and respond. The next one is false positives because remember the false positives and false negatives have an inverse relationship. So you actually want some false positives cuz if your false positives are zero, that means your false negatives are through the roof and you're missing critical information.

So you actually do want your false positives to be about 15 or 20% because that means you're reducing the false negatives. What is the average time to fix software vulnerabilities? How long are systems vulnerable and exposed? How long do you have critical exposure points? The next one, which is one of my favorite because almost every single attack, including solar winds was caused by systems visible from the internet that weren't properly patched that had access to information. They should not have access to. So patching, see how long does it take for you to patch your systems?

And here's the trick here? I don't want average.

I want, what is it gonna take to patch a hundred percent of your internet facing systems? Cuz the problem with most organizations is they only patch 90% of their internet facing systems. And those are the 10% that get compromised. So I wanna know what is the latency from when a patch is released to when a hundred percent of your systems are patched and then finally incident response volume. What are the number of incidents you're having? And this one is counterintuitive because the more mature you get, the more incidents you should have because they are happening and occurring.

You are just not seeing them. So as you mature and get better visibility, you actually should be increasing in incidents, but decreasing in response time. So the really mature clients that we work with, they have sometimes multiple incidents a week, but they're responding and detecting within minutes instead of hours, weeks, months, and years. So those are our five metrics. I I know only at about 18 minutes, I'll pass it back to our moderator for any questions or comments you might have.