A Ransomware Attack Against Your AD – The Things To Do Pre-Attack And Post-Attack!

Welcome, so welcome. I hope you enjoyed your lunch and not too sleepy to listen to my session. So welcome again session about ransomware against your ad and the things you can do pre attack and the things you should do post attack when the bad thing happens. So my name is George El Pinto. I'm based in the Netherlands working for St. Paris for two, almost two and a half years as a senior incident response lead.

So when customers are in need of help due to a breach, ransomware attack, whatever, they contact us and then I'm one of the people that will try to help them out and until today I've succeeded in helping those customers out, been working with security identity and recovery all my life for almost 20 plus years or something. So that means that I'm old, unfortunately. The topics for today, I'll discuss a few topics that are related to this session obviously to also make it a bit more interesting.

Also, one of the scenarios that we encountered at a customer, obviously anonymized and also some recommendations that you can take home with you and then think about and also try to apply for your environment. But first thing, first let's start with a demo. This is not a product demo. This is basically just to show you what automation can mean for you. So this is an environment that I have at home but three domain forest with seven domain controllers. It contains about 42,000 users that ad and all these servers that you see here are basically standalone servers.

They have just been freshly installed and they're going to be used to restore active directly onto them while I'm presenting. So I just initiate the recovery, then I'm going to do something else. This is a tool, it's already prepped to start. There it goes. We'll see what happens at the end of the presentation our are we back or not?

So one thing you need to realize many people talk about it is assume breach, but you should also understand that there are many people on the internet trying to get into your network and because at least for the things that we see, ACTO directory doesn't always get the attention that it should be because everything, everybody thinks we are safe, we are doing the correct thing, but there's always a hole somewhere in the ACTO directory and you need to continuously scan for it.

But then again, at some point in time after the attacker has entered your network and then presses the button, it's basically game over and then you may see something similar like this and when you see this, it's really too late 'cause everything, especially your ad, then the core identity system of your organization is probably burned to the ground. The next question is obviously what should you do? What can you do?

But, and before you get to this stage, it's also important question is what should you do to prevent, to even prevent this scenario? And there's a lot of things that you can do because prevention is even better than the remediation. Afterwards. You should actively meaning continuously search and fix, therefore assess the security of your active directory environment for identity, sorry, for indicators of exposures. Indicators of exposures are nothing else than weaknesses in the broader sense of the word within your active directly or your domain controls.

That's the whole thing and you can use different tools for, for this, you can either do it yourself through PowerShell, I've done it quite a few times, but you can also use tooling available that is freely available. Purple Knight for example, is a tool from SIM that you can use and it'll scan your active directory and your enter ID if you choose to do so for I think these days 150 plus security indicators, all kinds of things that are known, all things that are maybe even not known yet, but also kinds of things that we find out, hey, this is not correct, this should, this shouldn't be here.

And then we include it in Purple Knight so that when you use it, you will get a flag if you have that specific setting. Certify for example is a tool that you can use that goes a little bit deeper into into your certificate template configuration in your active directory. If you're using active directory certificate services. Grouper is a tool that goes deeper into your group policy objects. It gives you more insights into those settings. So combined all these tool combines gives you a very, very nice view how things look like. I call these the so-called static configurations.

Like for example, just a few examples because there are so many pass, sorry, accounts with password never expires or accounts for for the password is not required. Insecure anonymous log on, which is could be enabled in your active directory. All those things you need to be aware of and remediate. The other part is invisible attack path. There are two tools, at least to my knowledge, forest Rub and bloodhound. They at a high level, they do it the exact same thing. They show you the attack path from one point to another point. It's just the the the the the path. How that is defined.

That is the difference between the tools. Looking at bloodhound, it gives you all the attack paths and you need to categorize or prioritize which ones are really important for you to mitigate first through. It takes the other approach. So where bloodhound goes from outside in for through, it takes the approach from inside out, meaning you need to know your environment. You know what your tier zero is, therefore you define the default is the tier zero plus everything else that is for you to tier zero, that's your tier zero definition.

Then anything else that interacts with your tier zero, that's where you need to cut off. So you only need to pay attention to the things that interact directly with your tier zero. To give you an idea, what is a possible attack path? For example, I always say a default installed by default is insecure. One of the settings, for example, every user in active directory is allowed to de to perform domain joints, up to 10 systems. You might think who cares a tech path? Think about it.

So I'm allowed to add a machine to a domain, not really important maybe now let's put certificate services into the game. You might have a security template or a certificate template that for example is wrong configured, wrongly configured in terms of permissions where it for example has the permissions. Every computer in your ID is allowed to request this certificate. Sounds like a normal configuration. And then that certificate template for example also has something like allow the definition of a subject alternate name.

Now that's where it goes wrong because I am allowed to join a computer to the domain because I'm allowed to join a computer to the domain. I am an admin on that computer because I'm an admin, I can impersonate that computer because I can impersonate that computer. I can request a certificate from that insecure template because I can request a certificate from that insecure template and now here it comes. I can specify a subject alternate name.

I can basically impersonate anyone within the active directory, your ciso, your CEO, the main admin, whoever, just specify the UPN and then you can use the certificate for authentication game over. That's an attack path. All those things combined, those are there but it's not visible, at least not for us humans. So you need a tool that shows you that path. That's why attack person analysis is very important. In addition, you also have to look for ident indicators of compromise. This means if you see these things, things already went bad, think about DC shadow A.

So-called domain controller introduces changes, changes into the ad, mostly performed by mimic ads, Cooper's roasting requesting tickets and then brute forcing them offline while and then have as soon as you have the passwords. It's also game over many more. This was a lot about technology, but you also have to think about procedures, your backups, the solutions, creating those backups, but also the restores, periodic DR tests and many, many things.

You have to pay attention to the technology, the processes and obviously don't forget the people if performing security assessments is either too complex, too time consuming, you don't have the resources or whatever.

Some Paris from a, from a A company perspective has security assessments from both AD and and intra id, not Azure ID and ID where we get the data from the environment, we perform a security architecture review, we analyze the data, we write a report and then you get a prioritized report containing all the information that is obviously the good things but also the bad things that you need to pay attention to. So what can you do today on your own without a consulting company or us or anyone else but everything for yourself.

When you look at your id, you might look like it and see, well my ID looks like this and I would like to transform it to this. So these are cool pictures by the way. These were generated with Microsoft ai, so therefore not copyrighted or whatever. There was always a mistake in those picture. Look at this one, the stick through in the head of the docks, whatever really nice. So you want to go from one to the other. So you want to strengthen your active directory things. You can do tiering model. With a tiering model.

You basically segregate in let's say three tiers, tier zero one and two, where tier zero is obviously the highest privilege, tier zero for your service and applications. Tier two for your work sessions, you segregate the administration, meaning for every tier you have a separate account. Nobody likes multiple accounts but it's a unfortunately real life thing. So you need to have them. But in addition, when you create a tiering model, you should also take into account in hiding that data. Active directory is very powerful but it has one major bad thing. At least it's its sea fault.

It's open to anyone, any user can read anything and because any user can read and see anything, I also consider, well visibility is also vulnerability because if I can see where my high privileged accounts are, I know what to target. If I can see who the members are of powerful groups, I know who to target. So you need to, when creating the tiering model, hide those objects. It's possible.

Obviously you need to take into your into account that as soon as you start changing the permissions of that model, don't forget applications that for whatever reason still need to see that information, but the main goal is your default user population is not able to see who the highly privileged accounts are because that's where it starts. Somebody clicks on a link, something is installed and from there, there it goes. And obviously make sure to have a delegation model. Don't use the default T four groups because there are groups over there.

They might seem harmless, but from being a member of that, so-called harmless group, that person can become a domain admin. Therefore do not use the default T four groups that are still in ad lateral movement prevention use labs. It's from Microsoft. There's let's say the legacy version of it, which you need to install and extend the schema for, but you can also use the modern version of lab, which is already included in the operating system and in ad and I think it's from 22 and up after a specific update, it's already there.

The default domain admin account, that's an account that's always in ID in every single domain and it is a very powerful, very special account. If anything else breaks, that will be the only account that is able to log on. It does not have the restrictions that any other account has. This will work. I have a blog post where I have written down what you can do to use this account and you can even disable the account to prevent mis prevent being misused, but you can still enable it in a very special way to when you actually need it.

So it is very secure until you're actually need it after a specific amount of steps, procedures, resetting passwords of very powerful accounts. Script GDT default domain admin and DSRM stands for directive services, restore mode accounts. This is about procedures. Additionally, passwords. Now who loves passwords? Nobody. Everybody has passwords For many years we have made the mistake in telling users how long a password should be, how complex a password should be, eight characters, 10 characters, 12 characters. It needs to include us and exclamation mark.

And then you get passwords like 10, 1, 2, 3 explanation mark Crappy passwords. So turn the mind around and not static statically defining passwords but use password phrases and I like to play football. Maybe even a stupid mistake in the wording.

It really, really needs education for your users because most of the times they will ask how long must it be? And then it becomes difficult because it's not really defined In terms of passwords. Change the mind in if for service accounts start using machine generated passwords and store those in a password vault. Make sure they are not reused because password reuses is also an Achilles seal. Think about a normal person having a specific password and then has passwords for his service accounts and his admin accounts and they're all the same.

Compromise the normal user account and you have ownership of all the others within id. It is possible to define password policies for all kinds of types of accounts using the password settings objects and then it's also possible to prevent weak passwords using either enterra ID or a third party tool that if I'm not mistaken is still free, which is list net password protection that allows integration with have I been pond account hygiene. You need to continuously look at the account settings. If anything changes, clean it up. If it's not needed, get rid of it.

In other words, keep, always keep an eye on it. Certificate services I already mentioned when we perform a security assessment, I don't think we have had ever had a way where it was not included. There's always something wrong and we don't know why it, the only thing we can think of is apparently it's misunderstood and not correctly configured because certificate services is for sure a way to get into your ad and gain domain ownership. Have a look at those legacy protocols or security measures not being implemented. Have a look at those. Stop using desks, stop using RC four.

I know sometimes it's easier set than done, but think about it, try it. Stop using NT LM V one, preferably also V two.

Again, it's easy set than done, but if you don't try you won't get there and enable additional protections. Enabling the protection, the the, the, the, the additional protections could mean that applications could break, but then again you need to try and talk to your vendors complexity, believe it or not. But the more complex things are set up, the more difficult it'll be to secure them and if people don't understand what the setup is or the configurations, that's when mistakes happen and then you get security holes and that's when the attacker U uses them and go against you.

So think about in reducing complexity it helps you trusts. Well, many years ago Microsoft said the security boundary of an ad is the domain. Well I think a year or two later already, 20 plus years ago it was the forest and then we became aware, everybody became aware. If you create a trust you basically extend the reel and then it's not more the security boundary but it's the combination of the forests. Then you might think, well it's not a two-way trust, it's a one-way trust where for example, he's the resource forest.

I'm the account force meaning that he trusts me, I'm allowed to access him, but he is not able to access me. Hey, I'm secure. No you're not because the trust means there's an account on both sides that has a password and if I am an attacker that I own his forest for whatever reason, I can get a hold of that password which is the same on my side and then I can misuse the trust account on my side to get the information from my forest. It wasn't a one-way forest trust, which should not allow it, but it's possible to misuse it against you. Is there a way to protect this?

Yes, using authentication silos you can protect it with an easy configuration. I won't talk about it now, but if you are interested in knowing more, just contact me and I will tell you how Patch, patch, patch, patch, patch. Think about how to patch your systems because operating systems unfortunately contain vulnerabilities. I think one of the, one of the most common ones is the zero log on where you could just simply became a domain admin test backups.

I hope everybody tests their backups and testing backups is not only, oh, I created a backup, but can you actually restore the backup And restoring the backup is not, oh, let's restore just a few files. I've seen it happen where auditing said you need to prove your ad recovery and they restored a few files and yeah, that's done. But the funny part is that the auditor doesn't understand the technology so he accepts it as yeah, that's fine.

No, in terms of active directory, you perform a restore, a full restore of the domain controller and you see if it comes up. If it comes up then that's a successful restore. Not just the restoring a few files and obviously execute the R drills every single year, at least every year. Monitor for changes, monitor the security posture continuously because your environment is not static, it's dynamically.

Things are going on continuously so things will change for the good but also for the bad and you need to be aware of what's going on Now why you need to do this, well in many organizations and maybe even yours, but until today it's still the the case that Microsoft wants to have us all in the cloud but active direct is not going anywhere for many years and I think for a few years the commit won't go anywhere because it's the core identity system for many organizations Gain ownership of AD and you will probably be able to gain ownership of your intra ad.

If things are set up incorrectly, that will for sure happen. One thing to be aware of, although there is integration between the two, make sure that from an admin, from an administration perspective, there is no determination of one of the other. In other words, admin accounts should only live in the identity system where they perform administration. What I'm trying to say is don't have an admin account in your ad that's basically a nobody in AD but then synced to enterra ID and in Enterra ID is for example global admin.

In other words, I compromise ad get hold of the account and then I can authenticate against your enter ID and take over your enter ID game over. Make sure you have that split the other way around. Today is also possible if you enable cloud trust from enterra ID to ad it's even possible to attack your ID from your enter id, sorry, it's yeah, from your enter ID the other way around. Why? Because it's a default configuration. Microsoft set up a few things but they forgot about a few other things and it's easy to misuse that in another word ad is really old.

AD was built not for the time of today. It was built to withstand all kinds of localized errors, outages, networks, it's sorry, data centers but not for the world of today because today's attacks are about networks. Just attack it from the network and there it goes. So it was never built for today's world and the choices you may have made in the past may not and and they seemed okay in the past, they may not be okay today. Have a look at those things and get rid of them if you don't need it anymore. It's like basically clean up your room.

If you don't, at some point in time something will happen or at least somebody else will misuse against you. Business reasons, just a few points. Why do you need to do this? Regulations tell you you have to have a disaster recovery plan and you have to work on it, pay attention on it. Complexity, obviously there's always a, let's say difference between thinking in management and technical people. Managers might think hey ad restore or has just restore a few domain controllers, restore a few servers and I'm back. No AD is a special beast.

You have to restore the domain controllers, the servers, the the servers that carry ad and then you have to perform additional restore service, restore steps on the service itself. It doesn't fix itself by default. You have to do that. Therefore it's more complex than just restoring a few servers. Risk management, for many years nobody cared about restoring ID because nothing would happen unless some administration mistake occurred, but these days it's about external people, attackers, hackers all over the world trying to get into ERD.

So the risks have changed, the risk is higher and it's more like when than if. So how acceptable is it when your active director is done? Many times, ah, it won't happen to us until it happens and it's like, oh dear, we are in deep. You know what? All kinds of things that you need to prepare for. As I mentioned for many years I was talking about these scenarios. Sure they obviously they occurred, but these days these occur, these scenarios occur continuously every single day. A company or more companies are obviously attacked, but then again, it doesn't mean that the other things won't happen.

About a year or so ago, a company in the Netherlands, I won't name its name called me. Hey George, we have an issue with ad, it's still running but all the domain controllers have a corrupt database. Oops. And then we found out after we tried to help them that the backups they were creating were backing up the corrupt databases. So we had to go back two months in time because that's when the corruption, we never were able to find out how the corruption in the database occurred, but they had to go back two months in time to get a healthy database and therefore a healthy ad.

Now when you are down, you have a few options and one of them is rebuilding from scratch. When you talk to security companies that let's say are not ad savvy or aware or the possibilities, they will tell you you need to rebuild from scratch because you get a new ad, the attacker has no credentials, everything is pristine, you're fine. That's true. But Rebuilding from scratch means reinstalling everything, reconfiguring everything, recreating everything and everything. I mean all your users, all your groups, all your applications, everything.

Don't forget about enter id by the way, same story. So is this the correct moment to basically install a new id? I don't think so. Very wrong choice because at the end it's about trust. I understand that part.

You want, you want to get the highest trust level from your ad, but this is the wrong moment. As soon as you are down, my my thought is you want to get up as soon as possible to continue your operations and then make specific decisions. What's the under option recover from backup. One of the things we recommend restore active directory in an isolated environment. Why any isolated environment? As I mentioned, active directory. When it's attacked it's from a network perspective, so how do you know if the attacker is still there or not?

You don't unless you have for example EDR teams that are continuously monitoring that stuff. So that's why we suggest create an isolated network or your domain controllers may be set up in such a way that the network where they're where they are in is let's say possible to isolate in a certain way. That way you can work on that environment without interference from anyone except from you and your colleagues. You use always the most recent backup. I will talk about it later more why the most recent backup you assess its security. Just restoring is not enough.

After the restore, assess security and then fix the most important, the most critical things in your ad. Categorize and fix the most important. The other stuff is for later talk about it later, this is what we recommend. Why less downtime? Trust me when you have to perform a forced recovery as I'm talking about, there's no fun at all, but it's the best thing you can have for let's say the moment that you're in and afterwards you need to get, you need to fix the other stuff that's more important but it's faster, less downtime, less impact for your users.

You do need to have preparations in place like creating isolated networks, installing new machines because as my A ads is being recovered, I need to restore onto something fresh, installed machines. Be very careful where you get those machines from because if those are infected, so fresh installed is is let's say the best way and again, keep it simple. I've seen some crazy DR scenarios where the complexity was so crazy that it just gave me a headache and trying to understand what to do and how to do it.

And my thought is if I already have to think this hard to even understand it, imagine the people that have to work with it. So keep it as simple as possible. Don't use all kinds of exotic stuff unless you really, really, really need it. And this is the moment about trust. So in the other one, you install a new ad, obviously you get trust, but again, it takes a huge amount of time. It can take months for before you're back and in this case after recovery, then you can ask yourself the question, do I have enough trust in the current ad to continue to use it?

And if the answer for whatever reason is no, then you could say let's migrate to something new again, migration is also not fun at all, but at least your environment is up and running and your business is running. This would be the correct moment to say let's go for a new ad. Disaster recovery plans. The options. If you go for let's say the Microsoft default as I call it, there's a link on the internet with all kinds of information, how to perform a disaster recovery ad and let's say that this is a summary, the link is at the bottom, but this is a summary of what you need to do.

All kinds of steps and again, it depends the frequency of the steps depends on if you have a multiple domain, single domain and obviously if you have multiple domains it's going to be painful. The biggest environment that I've ever heard of, I'm not saying it is the biggest environment, it's the one that I've ever heard of. It's a monster 325 domains in a forest with 1100 plus domain controllers. Now that is a headache. Imagine doing this manually.

The other way, and this is what I've done before, is semi automating it, automate all those tasks as much as possible, but think about it, this means that I actively need to do it or by tool. You have to think about how much investment and time does it need to create those scripts and tools on your own, maintain it, sharing nodule, yada yada, the whole thing or just buy a tool. Nothing is for free but you have to weigh the costs, the pros and the cons. This is what's happening in the background right now. Let's see the differences.

As I said, from left you've got the Microsoft default in the middle, the the semi-automated stuff and at the right, the far right, the F fully automated stuff, maintenance if you use it, the manual stuff, it's more difficult. You have to have a lot of knowledge. The further you go to the right, the knowledge is more into the scripts, into the tools. It doesn't mean you don't know how to use them. Obviously the winds, the how, et cetera. But the more you go to the right, the fully automated, the less you need to know because the tool will take care of that.

Going back to the, the first point that I almost forgot to to say is when I built those scripts, my biggest issue was transferring that knowledge to my colleagues. I had all kinds of knowledge because built throughout the years, how do you exchange that knowledge built throughout the years including all the nitty gritty bets and good details in a month or so to other people it's going to overwhelm them and then when they actually have to do it, they're like, I don't know what to do. Call the guy. Many tools look at automated backups. Everybody creates automated backups.

But what about automated restores and not just a restore of one machine but for example a complete ad environment and also the com automated service restore on top. So if you can see backup is automated, restore is automated and the service is also automated. Lead restored the size, the left on the right and then in the middle you may use like system state backup, which in Microsoft terms basically means the complete operating system, the tool we have, we only care about ad, we care about the database we create about the ful. Everything else we leave behind, we don't care.

The important thing about it is any ransomware malware on the operating system, we do not take away. We only take the ad data from those systems. That's what we restore because if whatever you back up is what you're going to restore, it's a fact. Looking at again the left and the middle.

When you perform a restore, obviously the middle one, it depends on what you do scripting, but most of the times it means you're going to restore one domain controller per domain, Fix whatever needs to be fixed and then you clone and redeploy all kinds of manual tasks may be supported by some automation on the left, a tool where you're not the only ones, there are others. You perform the restore of one multiple main control, those domains, whatever you want, it just works.

Obviously looking at all this, the speed, the further you go to the right, the quicker it's And very important, as soon as you have completed your restore, you have to fix security. You cannot say well you can because it's your choice but you should not say, oh let's go back to production because don't forget, whatever you are restoring is by default weak because that's why you were attacked. That's why your ad is down. So you restore, fix the most critical stuff, then you go back to production. It's tempting to go back to production as soon as possible.

But let's say that you don't do what is needed and the attacker is still there a week later you might be doing the same exercise. That's something you don't want. You want to do it correctly the first time. A lot of other things, there are a lot of things you have to think about. Communications, logistics, plans, codes, scripts and tools. Where do you store those?

When you need to store stuff, make sure you store things on environments that do not depend on ad because if you need ad authentication, enter ID authentication and everything in the end depends on your ad and you cannot access your stuff because your ad is down. Oops. And if you do have a system that has a separate authentication mechanism for example its own, make sure you always monitor that whoever manages that system does not suddenly change the authentication system from let's use AD or Azure ad. I've seen it happen at a customer.

It was separated therefore logging on username, password, fine, strong password obviously And then out of nothing one day, no more request for credentials. That's weird. Let's just log off, log on again.

Again, no request asked around. Yeah, we integrated the the the authentication with AD so that we get SSO oh that's when it will hurt when time comes and then we got the advice.

Yeah, store it in teams, nevermind. No, there's no point because that depends on Azure. Azure AD may depend on your ad. There you go again. So think about where you store your stuff, credentials depending on the the the tool you need, think about the credentials. You need your default domain administrator account, your disaster services, directory services, restore mode account. Are you able to get them? Are they in a vault and does the access of the vault depend on ad? Make sure it doesn't at least for those accounts.

And the fourth point basically tells you make sure to have a look authentication and authorization When your ad is down make sure it does not depend and the only way to know that is to actually perform those drills because then you will find out what is wrong and then you have the time to fix it when you actually need it. That's not the time to fix it. And one thing and I will talk about it a little bit later is when you restore your active directory, there will be, can be depends on the scenario, an impact on your intra ad because your a intra ad remains as it is.

Nothing happens, at least I don't, I hope it don't but your ID goes back in time. So where your ad in general is the authoritative source for Azure ID or enter, ID now suddenly enter ID knows more than your ID itself. That's something you need to fix. Also authentication. If you are cloud native company, no you're good to go, no dependency, but if you do have ad, you have a hybrid identity and if you're using password hash syn, you're good to go because there is no real time ad dependency ad goes down, you can still authenticate against your enter id. Why?

Because the password hash not the password. The password hash is synchronized through enter ID in the end. The users can still use that same password. It's just from a technology perspective it's how it's being synced. It's not the password, it's the hash.

However, it's also possible to use pass through authentication and that's because many companies are saying I don't want my password in the cloud. I want ad to take care of it. Which is fine until ad goes down, no authentication, it has a real time dependency or ad no ad, no authentication for Azure AD federated system. Same story could be A DFS, could be PingFederate, can be anything. Again it probably depends on your ad. Ad goes down there goes your authentication. Is there a way to fix it? Yes there is implement password hash syn in the background as a backup mechanism.

You might not not use it directly but you are synchronizing in the background those hashes to the cloud. What's the main effort with two and three compared to one? One If your ad is down you cannot authenticate immediately with two and three you have to disable those primary authentication mechanisms. So pass through authentication would need to be disabled so that it becomes native authentication. And for federated authentication it means you have to convert your domains from feder to managed and managed basically means native authentication.

Additional steps, try it out in a test environment because I've seen where thing happen be when you have for example identity protection, suddenly identity protection kicks in and starts to behave weird. Make sure to test it so that you know what could go on and if it goes on instant response scenarios, how we have let's say categorize them in our way of working, this is what could happen. Active directory is trashed. Azure ID is trashed. Well active directory is for example burn to the ground but enter ID won't be burnt to the ground.

This may more the content of your tenant that's deleted, destroyed, whatever and less worse, still bad. Your ID is burned to the ground but your Azure AD remains as is. So in terms of AD and Azure ad, which one do you recover first?

Well my belief is recover the data first if you have a tool to recover that data because it doesn't happen by default, especially not for groups contacts and computers for users there's a recycle bin but again, you still have to do the work, recover the data in Azure ad first, then after the recovery of your ad, perform a gap analysis, fix whatever you need to fix and then you enable synchronization. That's the whole thing. At a high level it's a very long and complex story, but this is at the high level first Azure ad, then your ad. This is a scenario we're going to focus on.

Active directory is th trashed and nothing else. These are the scenarios we have categorized throughout time and what we have found AD is attacked and still running. It's attacked, breached by some attackers, some attackers doing something. You find out about it and you try to fix it. When we are called, and this is a scenario when non-customers call us for help at this point in time it's always non-customers that call us for help. The first thing we do, we install our solution to create a backup as soon as possible. Why?

It's faster, it has less, it requires less storage and it's also faster for recovery and we can easily move it to isolated network. We install a change auditing solution that so that we capture all the changes that are going on in active directory. Why we gain visibility of what's going on? Is it your admin people or is it an attacker doing all kinds of things? We perform an active directory security assessment, not the one that I was just, well it's the same one, it's just the output is different.

We create a categorized list where we say short term, midterm, long term and the short term stuff is that needs to be done as soon as possible. Midterm right after, let's say going back to production longer term. On the longer project We try to fix whatever needs to fix and then you would be good to go assuming that the attacker doesn't have additional credentials, this one is more risky. The other one is basically the same one with an additional step. Same story breached, create a backup, we installed a change management solution or change auditing solution.

Perform a security assessment or in a, in an isolated network by the way. We move that to the isolated network. So you have two parallel ads. One that's running in production, one that's running the isolated network. The important thing over there obviously is no changes at all because we want to keep changes at a minimum even if preferably to nothing because as soon as you get changes then you have to fix those changes after you go back. This is what we've done quite a few times already.

We performed the restore in the isolated environment, change all the things, solution security assessment, fix whatever needs to be fixed. We then shut down the production ad and put the other one that was restored in the isolated environment back the other scenario AD is attacked and down.

Again, this is for cus non-customers that call us and then we have to deal with all kinds of scenarios where they're not using our tooling but we still have to recover them and we have done some crazy things using either their backup solution from whatever vendor they have or no, we don't have a backup, we have VMs or snapshots or you name it, whatever. But at least it's better than nothing. It's not supported, we know but it's better than nothing.

After making sure that ad is back up and running, we create a backup with our tool, move it to the other network that's we perform the restore, the cleanup, everything else and that's what's then going back to production. Which backup should you use? There's always a very interesting discussion. AD is burn to the ground. You have a backup of yesterday its security might be Eh, Not that good and you have a backup of let's say 10 days ago. Now which backup do you choose? The one of yesterday which is let's say fairly recent lower security, security posture, that's the better word.

Or the one from 10 days ago that has a better security posture but it may have not all the changes because 10 days have passed. Obviously the 10 days depends on how big the organization is and the churn of changes. Well whatever backup you choose, going back to production is not just recovery, it's recovery plus security. So either way you still have to assess the security after recovering that backup and then you have to fix the changes between now and when the backup was taken. So look at the amount of changes between today and yesterday or today and 10 days ago.

What is important here is, although the security posture might be worse from yesterday, in those 10 days all kinds of things happened. Users were created, users were deleted groups, group memberships, and one thing that basically happens in the background because of time passwords, Those are really painful because you have to distribute 'em, you have to make the rejoin. So what I'm saying in this case is always use the most recent backup because the less changes you have to care about, the better it is. The security still needs to be assessed and you need to fix it.

Cleaning groups, creating new admin accounts, all that stuff. Fixing security operations in ad that is easier because that's really ad focused. All the other stuff, if you have let's say created a thousand workstations for the past week, you have to rejoin all thousand if you use the the old backup. So my recommendation is always use the most recent backup no matter what and then fixed security, hybrid identity infrastructure in reverse. This is what we do today. You've got AD and you synchronize all the stuff to Azure ad or at least enter id. I like to Azure AD embedded but okay that's me.

So ad's authoritative for enter I id but your ad goes down, you perform a forced recovery and then you have less objects. So you now have to use intra ID to see what is missing in your Azure, sorry, in your active directory and fix whatever needs to be fixed. Let's put 'em a little bit more detail on it. Same story objects synchronized, everything is okay, your ad is attacked, you perform a forest restore, Perform a forest restore forest recovery and all kinds of objects suddenly are back again or they are lacking because everything was done after the backup that was created.

If you would just now re-enable synchronization for either connect sync, which is the on-prem solution or cloudy, which is the cloud solution for Microsoft synchronization, you might find out that objects in enter ID suddenly are deleted because it's not the the the the the that didn't occur in the deletion on ad. It's the absence of the object and your synchronization engine. Either one is like, oh, object's not there, let's just clean it up. There goes your data from a user perspective looking at the recycle bin in your Azure id, you can recover that as if nothing happened.

It's still impact because the object is into the recycle bin. You have to restore it. But if it's a group or a contact or if a computer today that information will be gone forever. You might think who cares?

Okay, let's give an example. Think about a group that has all your users that distribute licenses and that group for whatever reason is deleted. Suddenly complete user population lost their service to whatever thing it's issuing licensing for and then because of one group, suddenly huge amount of people have a serious impact and the only way to recover that is to recreate the object and reconfigured for whatever it was configured to do so.

So after performing a force recovery, I basically even before make sure that synchronization is not enabled, disable it And by disabling it, I mean for connect sync, disable the scheduler on the connect sync servers for cloud sync, disable the configuration.

It's very specifically because do not interpret this as delete the configuration or disable sync at all in Azure because as soon as you start disabling sync or deleting the configuration, you lose all the information that you need later to fix the things in ad because if you disable that Azure SD is like, oh, all the synchronization, the the information that was synchronized from on-prem, I can get rid of it. And that's exactly the information you need to fix things on-prem. So be very careful what you do.

So you perform the gap analysis based on what is in enterra id and then you recreate whatever is missing mutable id, it's a very technical story, but mutable ID is basically the glue between the on-prem object and the cloud and the cloud object and therefore it'll basically rejoin again together. That's what I'm trying to achieve so that the objects are not deleted and then you should be good to go.

Again, automation is very important here because there are many, many, many steps, especially if you have to deal with lots of data. A scenario, real life scenario. What we did with a customer, they called us, we are under attack and they were, the first thing we did was create a backup and immediately without even knowing, we performed a restore of that backup of that ad in an isolated environment. At that point in time we never even knew we were going to use it or not because we believed scenario one, we're going to fix this as it's going. That's what we believed.

The backup obviously smaller, faster, whatever to know what's wrong. We performed a vulnerability analysis. We used a few tools purple nights from St. Paris back then we also used Certify and we also also blow down because back then forestry wasn't available yet. And with this we had the static configuration but also with tech path. And this is what we found. Remember the tech path example that I mentioned? We found it here.

All the main computers, oh whatever, all the domain computers were allowed to change certificate templates so they could not directly get a certificate but they were allowed to change it as soon as they were allowed to change it then you can change it to your liking and then do whatever you want to do. Really, really tricky. And this is, well you might laugh, but it was there. This customer had a help desk account And its purpose was to reset the password of all users. Sounds feasible, but we found exactly what I just told you. But we also found it the other way around.

The everyone security principle in ad which is a security principle was allowed to reset the password of that account. In other words, everybody was allowed to reset the password of everyone. That's a lot of self-service. But you can imagine the damage that can occur here. You can basically get everything, oh look, a pink pelican. Never had that one in my session. Hi. And that's when we then find out through the EDR team, not one attacker as we initially thought there were multiple.

And that's when my colleague and I that were working on this look at each other and we were like, oh no, we will never win this fight because it's like sitting in a boat with so many holes and we don't even have enough fingers to put in every hole. As soon as you fix something, somebody else will break it. So that's why we made the decision, let's recover. Go back to the alternate recovery thing, but what we also did in the production environment, we said a change freeze.

Sure the customer doesn't create all kinds of objects, but we also had to stop, prevent password changes for computers and for users too technical to describe how, but it's something that's important for later on. We hardened ad in all kinds of things after the security assessment, tiering model, new admin accounts because trust is very important. So every old account was disabled straight from its permissions. We fixed all kinds of things that we found and then D-Day, we shut down the production environment, literally shut it down after the customer, make sure it never comes up again.

And then we basically moved the the ad from the isolated environment to the production environment. And obviously when you do that, although it's the same ad From a secure channel, that's how let's say member service workstations communicate with active directory and the bank controllers and Gerber's tickets. We have to reboot those machines so that they refresh everything and get the tickets and the secure channels from the restored ad, not from the other one. Customer survived. This is what we want.

We want to win from the bad guys and that's why you have to continuously monitor and understand what's going on, being smart about it, making the correct choices at the correct time. Takeaways and recommendations. I hope throughout my, throughout my presentation I made my myself clear that being prepared is key. Not waiting until something bad happens. No be prepared for it because it, these days I can almost say it'll happen.

It'll save you headaches time and a huge amount of work, especially if if you, if you have automated it proactively secure your environment, monitor your environment with ITDR tools, identity, threat and detection and response tools near real time Reactively. Be able to recover. Don't think about your active directory only. Think about every very important system like if you have a DFS or other federation system to get its configuration. If you have connect sync to get its configuration, it doesn't happen automatically. You have to do the effort to even create that backup or export it.

Especially for example with connect sync. If that system is also burned to the ground and you have fiddled around with its synchronization rules, how do you know how to reconfigure it correctly when you need to go back? Because the synchronization rules basically determine how and which data is synchronized from AD to on, from to Azure ad. And if you make the correct deci, the incorrect decisions on the incorrect configurations data might be destroyed because it's not default.

And obviously what we always say is make sure that if you don't know things or you you need additional help, do it before the attack and breach. Not well, not necessarily after because then you'll for sure need help, but it's better to get help before the attack. Very important, make sure to have password has synchronization enabled as either primary or the backup. It will be your savior when things go wrong.

Important also as I explained, when you recovery ID no matter what, perform the gap analysis, make sure whatever is wrong in your ad compared to your Azure AD and fix whatever is missing. It's a very detailed and complex thing to do. So I won't even go further than that and automate as much as possible. Automation is really key.

Yeah, yeah. Procedures. Validate your backups, validate your exports and your resource. Make sure it actually works. I've seen it happen with a customer, the one with the database by the way, that when that point in time came to perform the recovery, it did not work. The time they needed it, it did not work. And guess what? They never tried it out. That's the wrong moment to find out. It did not work. Make sure it works. Perform the yard drills. Integration is also a tricky one. Be very careful if you integrate products, How the integration occurs.

Because if you integrate products in a certain way, in a certain way, you might be able to compromise one system and then hop to the other system. For example, looking at our tool, it supports ad integration. What we always say, or at least what I always say is make sure whatever backup and restore solution you choose for your ad, it is ad aware, meaning it knows how to backup, it knows how to restore, but it's not integrated with your ad. Why not integrate it, attack your ad, then you won't be able to attack your backup system. That's why it should not be integrated.

Be very careful with those integrations. Don't cross contaminate Tier zero systems one with the other. Let's have a look how the restore is. Did it succeed? It is done. So all the systems have the new background, meaning they're now domain controllers and there's the ad, at least one domain. There are still two others. But this was basically the message that I want to send out and in this scenario it was done in 28 minutes.

Well, while I was presenting. If I look at the other methods, I won't be able to do it too because then I need my focus on the whole recovery part. Any questions and time is up Of all. No. Do you have any questions? Anyone? Enterra id Do you have to do almost the same steps or the as well the tools to to scan everything to back up and so on? Well in enter id, you don't have to backup up the infrastructure.

That's the main difference obviously with ad, because with AD you have to back up the infrastructure and by backing up the infrastructure, meaning the doming controls, you're backing up the data. In terms of Azure AD or Enterra id, you only have to back up the data to make sure that if an attacker gets a foothold in your enterra ID and is really bad, it starts dele deleting all kinds of stuff. How do you recover?

But then going back into your enter ID might be a painful thing, but because these days, or at least today, Microsoft doesn't really have a solution to, let's say if the attacker has ownership of your enter id, the only way to, and you don't anymore because he kicked you out. The only way to get back in is to call Microsoft and that's the, that's Isolated.

No, well if your answer is can I recover AD onto your Azure AD by using virtual machine engineer Azure, then the answer is yes. It really depends on what what you mean.

But yeah, so if you have additional questions, time is up, unfortunately. Sorry about that. If you have additional questions, feel free to email me. My contact details are over there and I will answer you as soon as possible. So thank you very much for attending my session. Thank you. Thank you.