Panel - Threat Hunting: Making Data Informed Decisions

I think we can actually skip the round of introductions before, because you just all been here already on stage. So maybe you could just start with saying one sentence, like, so what's your view on threat hunting? What's your vision of this whole topic and which we would kind of throw into the discussion? I think my one sentence on threat hunting is a, that it's slightly different to what you described. And for me, it's building hypotheses and then gathering data to prove or disprove these hypotheses happy to quantum more detail later, but that's it for now. Okay.

And now, so I just talk about it. And for me, attending is also to, to look for current intrusion and also to look for potential Tsion in the internal and the external parameter and together to have some other statement of how can I detect threats besides my security controls. Okay. Okay.

Well, I, I had a definition in my slide deck before in my talk and well, it's two definitions. One is finding the needle in the haystack, but actually the definition I had is that it's a process of proactively and ITER iteratively searching through your network to detect and isolate threats, which evade existing security solutions. So it's kind of both what you guys set. We need data.

So it's, it's a process, it's a manual process. You need to do it.

And, and we need data to, to, yeah. To, to get the, all the information we need to do educated gases.

Let's, let's call it this way. Can I ask a question to my fellow panelists?

Sure, Sure. So when you say the threat end is a process, do you think that process can turn into a detection rule or signature for Example? Absolutely. Absolutely.

So, well, I've also mentioned in my talk, we, we are short on, on cybersecurity, on threat hunting skills at the moment. So we do not have enough people to do the job, but we should have. Right.

So, oh, we, well, let's, let's not say should have, but we, we need people to do the job. And, and what you said is to, to have a machine doing the job because of, of baselines created front. So yeah.

It could, could be a way. Yeah.

And, okay, great. So I, I think we already had a question It's getting interactive.

Hello, This kind of subject, the tragic hunting it's it's it's new. My question is we have a more analysis. We have AOR and I think it's good to clarify what difference these three topics. I think it's good that here. Yep. And I'm happy to start talking about it. That's also why I'd not fully agree with you. I think there's a need for threat hunting, but I think it's madness to try to get threat hunter into every company or create skills. It's so specialized. It doesn't scale. That's why companies like yours exist. Right. Right. Because it doesn't grow on trees.

So I think there's a need for it, but there's also confusion around it. Right? What's the difference between a blue Tema, a red team are now threat hunter. So pen has red team for me is quite obvious you attack stuff, right? You try to break it so you can build new detections. Like you said, blue team for me is analytics driven, how you described analytics, driven. You sit in your sock and you wait for alerts to come in. And then you work through these alerts and a playbook for me, a threat hunter is separate to both where you don't wait for alerts and you don't break things.

You sit down tabular, RAA at whiteboard and think, okay, maybe based on external threat Intel or the diamond model, for example, for threat hunting or the threat intelligence permit, permit of pain. And you think, okay, I'm Deutsche bank to continue your example. I might be targeted by a malware actor that uses RDP to move literally.

And then, then with a hypothesis, I go into my data. I might deploy a telemetry in your senses. I might look at my existing solutions like Azure or Sentinel or whatever. And then I try to prove or disprove this. So it's a different process for me where you don't start by detecting things, but you start clear and then go and dig deeply into the stuff.

Yeah, I think I, I really agree with your approach. And I think that it is a hard question, how to define the capabilities of what is the skills that I need to look for in potential and what should I write in the job description when I looking for new employee. And I think that think things getting brighter from day to day, and as I see it, I believe that threat hunter should be aware of all the security systems and all the, the vulnerabilities. And it should be the one who collaborate between all the other teams of the security operation.

As you mentioned, the red team, the blue team, the threat intelligence and the forensics in order to find what inside what is going on, like I'm, as a threat intelligence will release a weekly landscape report about recent attacks for the financial sector. And they will be the one who take the findings and try to implement it in the way that he looks for these talked threats in the, the multiple clients that we are having. That's how I see it. I believe that it will be, be like very flexible during the time.

Cause as you mentioned, there is no strict explanation of this role yet, but yeah, Well, what, what I said before that threat hunting is a process, a process of searching and, and we agreed that this is a process a machine can do for us. The threat hunter today maybe does not need to be the person to know all the security issues we have in, in all the operating systems. For example, we have there, but there needs to be a tool that helps you to, to, to drag it down.

This is what, what we've done with the demos before, because with, for example, with this intelligent security graph we have there, that would be such a, kind of a machine learning tool to, to give you the, well, the, the information, the necessary information you need to, to react on something. And so a threat hunter could also be a person in the sock, be, be a person in the sock who, yeah.

Who, who is sitting there, who's knowing about the environment, but who's not the guy who, who has to, to really dig into it very, very deeply because, because it's almost impossible. I mean, they're specialists for window service specialists for Linux service network specialists, wherever. I don't think you find one per well, you might find one person, but they're really, really, really short and supply. And those actually exactly the thoughts or going through my head, where do you find enough threat hunter? Because it's, it's such a really totally new set of skills required. Absolutely.

Or like who, like, what would you envision such a next gen hunt would be, would be like a general having a lot of underlings doing all the men jobs for him, or would it be like kind of an artist psychic maybe, or be, or a professional only expensive one, which you would just hire on a subscription basis To jump in there. In theory, my title is director of threat hunting at my workplace. So I'm heading up 30 threat under. So I think a lot and hard about this and it can be very much aided by a tool you were asking, what does a threat look like? How can you nurture them? Where do they come from?

Our people in my team, which are mentor, they don't have it backgrounds at all. They sit in front of a solution like dark trace or Sentinel, for example, where you have the initial hypotheses, the leads that could be anomalies. So threat hunting is about embracing uncertainty in a blue team, sock. You hate uncertainty. You want your well defined rules. You want as little thought positives as possible. A threat hunter embraces uncertainty where just says an unusual amount of traffic going to Russia.

It's a perfect starting point for a threat hunter to D down and look at this stuff, but that's a different skill set. So we had loads of success using clever young people. No it backgrounds, but they have got a PhD in astrophysics, a masters in languages, maybe bachelors in chemistry. So they're very good at covering data points, pivoting around data, but might not have the deep down it security background, which tells you covers event one to one free 2, 1 32 is windows event login value or something. So I think it's a different skill set there that's required. Okay.

Anyone else to add something? Or should we just go and ask the second next question. I think maybe I will add something short. I think that the real skill, or maybe a character, character of a good should be to be passion about doing actions. Cause he can like, you know, he can be a passive and way to alert to come to come into the system. But threat should be someone who always need to be aware and do some checks. Cause actually his job is to discover new things so he can see it and drink coffee and be unemployed.

Cause you know, he is up to him, but we need someone will, will be look to fight. Okay. Look for a wall. So these kind of personal Like Albert Einstein. Yeah. Okay. I think we have a next question. Yeah. It's actually another question. It's a comment just on the skills required for the job. And just to add to that and you know, you touched on this as well. There are an lot of highly skilled investigators studying arts. I don't think we really require technical skills and cybersecurity as much as people like to think the tooling is always changing.

The, the job rules are always changing and I think you'll find in historians and archeologists and linguists and everybody else, investigative troubleshooting, problem solving, you know, thesis proposing type of skills. And, and we aren't investing in those people enough in our industry. And that's why we have a skill shortage cuz we do still retain this snobbery about technical capabilities.

I, I completely agree. I couldn't agree more. My 30 people have 50% women or 60%, I think huge diversity. And it's this mix of skills that really leads to this uptick and clever bright investigations. But I think that's only possible to have these smart young people. If the machines do the heavy lifting, you couldn't put a junior, which are the arts degree, no offense in front of a SIM solution to triage hundreds of logs, right? You need something that does heavy lifting to a certain degree. It allow you to do the clever analytical work on top.

Yeah, I see. I understand. I appreciate that. I think you're not wrong. However tools can be, be learned more easily than people think as well.

And I, I think that analytics and creative thinking are something that we need to sort of really appreciate more because those sort of those people here are the glue between everything else are, are, what's gonna get us through you think so. And just one short remark from my side. So here we have a discussion between just put representatives of a very large companies with billions in revenue. Probably what about us? Like keeping a coal, which only has like 50 people altogether. Do we need to hire a threat hunter or can we rely on someone else?

Well, you know, the drama I'm banging, right? So I don't think you should need to. So that's why I'm not saying by clever AI solutions, right? I'm saying we as an industry need to move away from, we need to hire more experts. We need to train more experts. Absolutely. But we also need to change the way we do security. So you don't need to hire three threat hunters for 50 people company. You could either either consume or manage service. Right. For example, which is always good. If you trust your SSPs, it's another topic I suppose.

Or if your tools to most like 99% of the heavy lifting, free blocking all the stuff, then that's perfect. We're not there yet, but that's the dream, right? I minute later it's good. Okay. I have a question which is touching into legal discussions and also ethical and morale of people. When you do threat hunting, one of the things you will most probably do is to access data on the internet, on the dark web or wherever that is most likely stolen. Now downloading data, which is stolen is actually not legal in most countries.

So what kind of legal considerations have you done at your companies to evaluate whether you can do this or not or how you can do it and also what kind of training do you provide for your employees in ethics or morals in handling this kind of data? Okay. So I guess it's for me actually we, we keep, we all the process that we are doing with the dark web is also outsourced using a third party company, which provide us the tool automation tool that we can access an archive that was already called from the dark web, from multiple sources.

And then like we have a dark web Google kind of that we can trigger queries related to our client's assets. So we don't need to get manually into the dark web places. And these vendors, like the third party software of the dark automations are legitimate companies and they are align with privacy rules and all the legal issues. And in cases that we want to access manually to the dark web itself, we have a very strict instructions how to do and what to collect and how to use like your identity in the dark web.

And it's full course of almost a month, which I really like to do like to, to have with my employees. And yeah, it's another topic, but it's very interesting If I can add one thing, it's a really interesting question, but I'd say it's quite theoretical. If you think about, we don't even have the legal or policing capabilities to hunt all the actual criminals down and you look at the gray area where people do hack back and do probing OSN.

So there was the case recently on Twitter where somebody hacked back a ran somewhere, author got the description keys from their server and released them to the public. And I'm not sure if anybody from like the legal system in their countries following up on that to see if that hack back was legal because it did so much good. I'm not endorsing this, not at all. I'm just thinking there's, it's a really interesting yeah. Theoretical issue. But in practice, in my experience, it often gets brushed another table.

Well, not just hypothetical. I'm the guy who discovered the LinkedIn breach in June, 2012 and went public with it worldwide. And I downloaded more than 6 million Shawan hashes in Croton and got assistance from people worldwide to try to figure out where dust this leak originate from.

And again, we were able to successful successfully confirm that the breach is indeed real data from LinkedIn. There was no question about it. Now I have been asked multiple times over the years downloading, you know, me downloading that data dump without knowing the origin, then cracking the passwords and figuring out where was coming from. Did I do something illegal?

And I, you know, in another scenario I do recommend business as, as an example, to crack the passwords of their own employees, not to punish anyone, but to have a perspective of your current risk in your own organization, but in several countries in Norway where I live also in Sweden and Denmark, I have also got into legal discussions on the legality of actually using that. And from a GDPR perspective, government says, no, you can't do that. I think you're highlighting, sorry for talking so much guys. It's All good.

I think you're highlighting an important point there, which is bigger than the discussion we are having, which is for hostility or the lack of regulation from big companies against security researchers. Right. We often see these knee direct reactions where just on a report of big hack and they don't care about, or they're threatened with legal actions. And that's definitely a big problem where people like you just wanna do good, but to get threatened by, by the judicial system or by big companies, lawyers. So I think that's definitely a problem, right? What you're highlighting there. I am.

I can add maybe that in order to share sensitive information with the client, also, when it's related to them specifically, we have signed with the, with the customer itself. That is, it is, it wants this information to be discovered. And also the way that we are sharing it is in a protected way in order that if we are breached and we don't know yet, so it will not be affected. So I guess that if you have the confirmation, like it's, it goes good and we are not collecting any other information about not of our clients. Yeah. But it's a very, it's a conflict, it's a dilemma. Yeah.

And that's where a lot of bug bounty programs start. Right. The first bug bounty program that can have is disclaimer, on your website saying if you find any bug, email it to info kc.com or something to avoid these challenges basically By way. So since we had just had that magic world pop up in the previous question, do you actually get asked a lot about GDPR compliance from your customers About compliance?

Yeah, Not really. They just want effective intelligence stay low to them. It's very spicy sometimes.

So not, not a lot. It's a big topic for our clients. They often ask how can we help them to be GDPR compliant?

So not, not about if we are GDPR compliant, which we are, but how we can help them. So for the customers we talk to, it's a huge topic. They're afraid of the fines basically. Yeah. Yeah. That's what I mean. Okay. Yeah. But GDPR compliance is not only a threat hunting topic, but it's, it's what you said. Customers come and, and say, well, help us to be GDPR compliant from our perspective. So for example, how do we use our user accounts for, for, for our environments?

I mean, as soon as you have a user account in, in Azure ID, you need to have, have kind of GDPR compliance because it's, it's it's data, which is, which belongs to the user. Right. Okay.

So again, so we still have quite a lot of time for questions. So just raise your Two short questions. Maybe the first, whether you cover IOT and operational systems threats also that usually don't are not connected to, to SIM systems. And the second question is you, you addressed threats that are potential or new from the outside and also threats that maybe materialized within the network and produced some noise. What about APTs that might be dormant for a few months in the systems and they didn't do anything yet. The way my wife thinks about me, rightfully. Hmm.

Well maybe from the, the point of view, I mean, going, going into, into this IOT space actually means there is no security. I always have a joke, which is the S in IOT is for security. So at the end of the day, an IOT device is an authenticated device. So the device will authenticate against any background service. So what you need to do at this point is to protect this identity, which, which, and with any terms, so whatever it is, you cannot protect the device network from a network point of view.

But from the I ID point of view, Your question, if I've got it correctly is almost segueing into a bit of more monitoring the threat hunting. Cause you said you can't collect locks from an IOT device. As in cases, you can't from ICS device, just three control systems and you cannot install an endpoint agent in most cases. So when I think about monitoring in general, which is predecessor to threat hunting, you wanna collect some data telemetry, which in most cases you have three ways, right? Sorry for the education part. It's a bird up to my explanation. First is law collection.

SIM second could be EDR. So endpoint detection response. So come black crowd strike in the likes or NTA network traffic analysis, where again, we sit. So we have loads of ICS and IOT customers where you don't need to redefine the, the city of Las Vegas is a customer of ours. So we don't redefine what an attack against a smart traffic like looks like, how would you even know? Right. We don't define what a threat against a smart carbon dioxide sensor looks like. We see these things being compromised, but you can't collect blocks.

You need to find another way, which is often the network and then clever analytics like you guys use as well in the cloud. Or we use for the network to detect the anomalies and then dive into it. So that's that part. And the apt is another whole topic, which I cut down my speaking time for now. And maybe my colleagues want to talk about that stuff. Thank you.

I, I can add about the IOT and OT, ICS devices that if we want to look for, for like many customers that are doing some, maybe they have factories and things like that. And like many companies has these kind of devices also IOT. And if we like from the external investigation that we're conducting, we are looking for open ports or a known F well of this, of these devices in Showan senses any other tools that takes the panels behind these devices and provide additional information. And we look for as a outsider of the company, what can I find about my specific client from these tools, okay.

From this eh, callers and what, what should I suggesting in order to block the opportunity to exploit it? Cause many times in Showan you can find default passwords for many IOT that are as a open port or open access and can be misused by threat. And sometimes it's also provide the specific CV. And then if you look for CV plus plus exploit in gouge, you can also exploit. So things are very easy and also big companies and enterprise fall for this attempts Back to your PT question, you said, how can you catch an PT? Right? First I think APTs are probably not most people's issue.

Actually, most people should be afraid of a commodity man where the spa fishing from financially motivated actors. So I think there's a big misconception that many people try to protect themselves against the biggest. APTs not saying it's wrong to do so. I just think there's big misalignment in what you try to, what people try to protect against versus what they should protect against. The second part is that was a big revelation. To me.

There is no apt alert, anybody, any vendor who says I've got PT alerts, they are just telling you garbage, because if you wanna find an PT, so a human group, that's moving slowly over a month. There's no single alert to catch that. What you can and have to do is look across the whole kill chain. So every single step and look for the trip up.

Look, if the PT, maybe they move literally once in a month, but if it's anomalous, they never write a file over SMB from that laptop to the domain controller on system 32, because it's just unusual. You have to a C it right, alert it, and then quickly investigate and come to a conclusion. So you need to have many, many trip wires, so to speak, be it the later movement, be it, the quantum control traffic, because you're never gonna catch everything. So you need to make sure we catch something and then can latch onto it and quickly investigate and or threat hunt if you wanna call it that. Okay.

Any further questions? Okay. While you think, when I'm thinking about your next question, I have one as well. So when I could have, before this session, I thought I had a pretty solid understanding what threat hunting actually is. Now. I think that it actually, it's almost like everything is threat hunting now, monitoring and detection and threat intelligence there, any limit, is there any wall or you would say, this is where we stop and the rest is not threat hunting anymore.

I think it's a real muddy waters here because people say triaging or monitoring or investigations are threat hunting, right? That's such a fuzzy term. That's why I am also saying it's the product of approach of not starting with an alert. You could start from a IOC, you could start with a hypothesis and then you go improve and disprove the data. Everything else is probably just triaging. If somebody says with a threat hunting tool, they present you with 40 alerts every day. That's not threat hunting, that's normal triaging B team operations, right? So I agree. It's very muddy.

There's huge discussions on Twitter about this with a few people. So it's, it's a heat topic for sure. I have also something to add.

Recently, I came into an incident in one of our clients, internal systems, and I understand that there is a IP that's starting scanning, trying to do some D attacks against him. And from having this indication, like just doing some deep diving to the iPad and understanding that it was breached recently, and maybe the threat act was using the bridge IP to conduct attacks. I understood that I need to also observe this activity in other clients in a proactive way.

And what was amazing to see is that the power behind having an access to multiple clients enables you to have new sensors for threat hunting. It means that this specific observed incident in one client led me to find it in two others while Dell security system will, didn't alert about it. Cause like there are different security systems that we're integrated with and we found that there one solution alerted and the other one not.

And I, I, I felt that this is the power of like, for me, for being as an MSSP with, you know, sensors in different country, not countries, also countries, but in this, in different clients. And this example of threat hunting, like we alerted the client, listen, you were, you had these attempts, which led led to a real threat and you couldn't detect it. And as we tried want it, we found it. So I like it too.

And this is a nice example of why, what is the power behind being access to having access to multiple clients Coming back to your statement like 10 minutes ago, you said, should we as KC hire threat hunter? I think a good analogy would be, if you say, should we, as KC build a Ferrari, right? It would require lots of engineering, power, loads of knowledge, loads of training of people. If you start from scratch engineering from scratch the same stuff, you wouldn't, you buy it or you consume it if it's a service.

So if it makes sense, because you want to have the very best Ferrari or racing car, you might start and invest that kind of effort. But for most people that's a not feasible or B doesn't contribute to their business. So they wouldn't do it. They just try to consume it. It's a bit lacking that analogy, but I think you see where I'm going with it, right? Yeah. Yeah. Totally. Any final words for this?

Well, yeah, again, I think it's, it's not about the alerts. I mean, yeah, we need the alerts. We need the monitoring. We need all the information to be able to, to react on it.

But the, the process starting from this point, this is what actually a threat hunting means for me. So it's, if I have my monitoring, then this is operating. This is what I have, I need to have in every it environment, in every corporate environment and reacting on the alerts and, and knowing what to do with these alerts, this is when, when threat hunting actually starts. So it was what you said. Of course we can get 40 alerts from, from Azure security center telling us, well, there is there's yeah. An identity attack to a server.

And, and then you see there's some way around, around the, the integrated security stuff you have in there. And these are alerts, but you need to bring them in context. And this is what threat hunter just do at the end, I think, and, and then find out who actually was the, the attacker and, and isolated in, in your environment.

Okay, awesome. So to summarize what I have learned today, so threat hunting is a policeman, a detective, and a Ferrari and a little bit of AI underneath.

Well, thanks a lot to all of our panelists. Thanks so much for the audience for sticking up till the break. So we will have about 40 minutes coffee break and there will be some additional sessions in this room. So see you later. Thank Very much. Yeah. Thank you.