It's great. Hey you much, you know. So hi name is Andrew Newell, I'm the Chief Science Officer at ipro. I'm gonna be talking about what are the impacts of your ai and I'm gonna specifically talk about the way that AI is being used in terms of a type of attack called injections. So I'm gonna make in this talk essentially made up of two halves. So in the first half of this I'm gonna be talking about the threat.
So obviously over the last two years or so, there's been an awful lot of talk about advances in ai and I wanna just pin down essentially what has changed and the big impact that this is gonna have. I then go on, I'll show you an example of an attack so we can understand the sorts of things that we're up against. I then put that into the context of the, the threat landscape.
One of the things we see now is the threat landscape. It's highly complex, it's evolving very, very quickly. So I'll give you just a bit of context around that.
I'll then briefly go through the trends that we see about how these various attacks are being used in the wild right now. In that the second half of the talk, I'll then go through how we counter the threat and I'll express this in terms of three high level things that we need to do. The first of these is about the, the direct link in between identity and that the physical user. I'll then talk about the arms race and how we can use the evolution of attacks to drive the evolution of the the mitigations. And then I'll talk how we can maximize information.
So as I say, I'll just start just by going through what's changed in ai.
So there's been an awful lot of talk about churn to ai, how it's new, it's evolved, it's very, very quickly. We come very, very good.
However, in the biometric world, this has been around for a long time. We have been conscious of the the threat from generative AI for many, many years.
In fact, going back as far as 2014, we had a machine learning technique, which is known as a GaN, which is a generative adversarial network. And this was one of the big first steps forward that produced high quality output that was entirely synthetic. Then we saw 2018, we started to see the arrival of things called hay swap apps. So a face swap, it's a type of deepfake, it's the most common type of deepfake that you see around nowadays. Essentially what it enables you to do is to take a video of one person and put the face of another onto it.
So we started to see seeds arrive in about 2018, the year after we saw sort of first autumn motion models. This is, again, this is a other technique, enables you to animate a single image of a person. It means you can, you can take an image, make it look as if that person is speaking, saying something that they weren't. And so what we've seen over time is actually that the quality of these various techniques, it's been getting better and better. What we saw last year was a massive growth in the tools and the techniques around things like face swaps. So the world of deep fakes has changed.
And now we're certainly in a world where actually techniques like face swaps, the quality is extremely good and we're seeing this being played out other media as well. So in the general image space, we've got tools like, and we've got the new tool for video soa, which is O, which is open from opener ai.
Now in terms of audio space, we've got audio deep fakes, they're getting better and better and the quality has gone through and obviously text. We've got the whole world of actually large language models. So the quality in all these media, it's getting better and better.
But the key change that has occurred is as the quality improves, we go through the boundary at which a human can tell what is real and what is fake. And that is that the crucial SH shift that has occurred here. Now you can argue with each of these media types whether we're going through the boundary right now or whether we've been through it. But the key thing is as we go through this boundary, these tools and these techniques that come from ai, they stop being to things which are just of interest as threats to automated systems such as biometric systems.
But they become a threat to every process that is linked to identity. So whether this is automated light biometric, whether this is a manual process such as a video I ident type process. And increasingly we're seeing these tools be used as highly effective tools in social engineering attacks as well. So really the the relevance of all these ai AI techniques has gone from being something that is confined to something that is now incredibly broad. Now I'll just show an example. So I mentioned face swaps. So let me show you an example of, you know, a face swap.
So the video you can see here, now this is the real me and you can see that I've got motion in my face. You can see that I've got natural lighting and you can see, you know that the background is there. What we do is take this video of me and apply another face on top of it, which is gonna make it look like another person.
So if you look now what you see is that actually this is another face. Everything that is not related to that, the facial identity has been maintained, the motion has been maintained, the background has been TAM maintained, the amid light has been maintained.
Everything except for the fact the face itself. So this now looks like a different person. So what a face walk does is it combines the cues from one video and puts them onto a source face. Now the reason why that we are seeing this as the attack of choice right now is it gives maximum control to an attacker. They can create the video themselves with all the motion and they can apply it to any face they like. So all the faces which are used in this video are synthetic. So you can do this over and over again.
Essentially what this enables you to do is to create a video of anyone appearing to do or say anything at all.
Now as I mentioned, face swaps, they have been around for a long time, but in the early days you could spot by eye that these things weren't very good, that the quality wasn't quite there and there were flaws. But even now, even when you ask users to do complex actions like put their glasses on, you'll see that there's no breakdown in these techniques at all. Whatever the user does in a video buy eye, these things now look perfect.
So this is why as I say, this is now the stage we're at. This is no longer just a threat to things like biometric systems. This is a massive threat, you know, to video identity systems and these sorts of techniques are exactly the same ones that are being used in the very high profile social engineering attacks, which we're seeing right now. Now a face swap is a type of imagery.
So what we have to do is to place this in the context of how you turn that imagery into an attack. Essentially there are two things you can do.
You can take your deepfake, your face swab, you can put it on a screen and you can show it to the mobile phone that the webcam. Now in the biometrics world, this is what has been known as presentation attack for many years. Presentation attacks, they've well understood, they've been around for a long time and there's a very large body of actually academic on how to stop them, which is known as presentation, attack detection or pat.
Now changes in ai, their interest in in the context of pad, but they're not something which is a particular game changer because anything you present on a screen is still something that's shown on a screen. The real threat that comes from things like face swaps, DeepFakes, and any other thing is in terms of an attack that's known as an injection attack.
Now the crucial thing with an injection attack is because the image is not shown on a screen, the traits in the imagery that have previously been used for pad techniques don't work anymore.
Instead, the imagery that is injected, it doesn't have to be changed in any way at all. And this means that actually the vast range of imagery types that you have out there, all of these things have have to be counted in an effective manner. Now I won't go through the whole of the, the threat landscape here. It's very broad, it's very complex. But I just want to emphasize that in injection attacks there are essentially two parts.
There's the imagery that you inject, which might be in a deepfake, and then there's the way in which you inject, which might be a virtual camera and that in that the middle attack or you might tamper with the and and code in some way.
Just to give you a sense of the complexity. So I mentioned face swaps, which you'll see down here on the bottom right face swaps are just one type of deep fake. There are many different types of deep fakes. Deep fake is just one class of imagery which you can use.
And in an injection attack and even within something like face swaps, there are so many different techniques and tools that can be used for face swaps. Our analysts currently track over 110 different face swaps tools alone with all these imagery types. Each of these can also be injected in any of the various injection methods that are out there. And just as with image types, there are lots of different ways to inject.
I won't go through the whole thing now, but essentially when you combine different injection modes with different imagery things, you end up with thousands and thousands of different attack factors.
All of these are evolving. We are seeing new tools come out pretty much every week. The tools that exist, they bring updates out. So the crucial thing here is the threat landscape is now immensely complicated. It's evolving very, very quickly. And when we talk about countering this, we can't just talk about an algorithm to stop DeepFakes.
We need a comprehensive system that can monitor, assess, ensure that we can adapt to have an effective defense against this. Now you might ask me, you mentioned face swaps. Are these just a niche academic attack? Are these actually being used in the wild? Lemme just take you through a little bit of what we see in the world right now. So the trends I show you here, this come out of these come from the iStock. So the iStock is the I proof security operations center. It's essentially what we have to monitor the the threat landscape.
We have always operated on the assumption the threat landscape will evolve. This is an arms race. It's crucial to be able to understand that evolution at the earliest possible point in time so we can ever respond to it. So the information we have here, as I say this comes outta the IO. The IO essentially consists of automated alert and tools combined with a team of analysts who are expert in DeepFakes and injection attacks, who goes through and assess what we're seeing so that we can be highly confident of the trends that we show here.
So in terms of face swap ejection attacks, what we observed last year, H two over H one is an increase of over 700%. So 704% to be exact in terms of the number of face swap attacks against our end system. Now what we think is driving this, if you go back a couple of years, a face swap, injection attack required expertise to create a face swap, it required expertise to inject and expertise to be able to glue these techniques together and that placed a relatively high bar.
What we've seen over the last year, especially in the second half of last year, is the arrival of combined tools that combine the creation of the face swap and the injection process into one. They're highly configurable and highly easy to use. So what we see now is there are tools that you can download often for free inside an hour. You can be up and running, creating face swaps and injecting them with pretty much, you know, no prior experience of this. So what has changed is that the effort that is needed is very low. The technical exp expertise it's needed is very low.
And that's what's been driving these. As I say, we currently track over 110 different face swap tools. The most common one that we see is a tool called swap face, but we also see quite a lot from deep face live swap stream with samples from others as well.
I'll just comment briefly on our general injection attacks. So what have we been seeing? I mentioned here that these are evolving. So just to share a few trends we see here, again, if you go back about two years injection attacks with things that will launch against place it on the web platform against laptop computers.
It was mainly around virtual webcams. That's a piece of software that runs, you know, on a laptop or a desktop, allows you to inject the video and make it look alike. It is a webcam, but again, what we've seen is massive evolution in this space. We saw a massive increase in actually attacks against mobile web H two over H 1, 250 5%. We saw a massive increase in the use of emulators. An emulator is a piece of software that runs on a laptop or desktop, makes it look like a mobile phone, 353% increase.
And also, very interestingly we saw the arrival of a new type of injection tool and this is called a native virtual cam, that it's a piece of software that runs on a phone, allows you to take imagery from another source and makes it look like it comes from the camera on the phone. So again, this face is very complicated, but just the key takeaway here is injection attacks are very complex and diverse and they're the most likely route for persistent threat actor. Now this may sound very, very bleak.
I've painted a picture where I said that example of the generat, I like face swaps, you now can't swap them by eye. I said there are loads of different types, it's a highly complex space. I said they're being used in the wild right now at scale. So this picture does sound bleak, but there is an answer to this.
There's a technology based answer to this. We have to accept the world has changed, we need to think differently and these are the requirements that I think we need.
So just to come to the fundamental problem of what we're trying to do, which is to link a digital identity with a present physical human user. And this link between the digital and the physical is the absolute crucial one. Now as the threats grow, the direct link becomes ever more important. So what we want to do is to get the most direct link between the the identity and the physical user. The most direct link we can get is through biometrics combined with strong liveness. That is the way that we can create a link between the, the identity and that the physical user.
Now a crucial point to bear in mind is that not all liveness, two detection techniques are the the same.
There's some that create a more direct link than others. The crucial thing here is what we really want to do is to create a direct link between identity and a physical user at a moment in time. Now to do this, we need to use what is known as a challenge, a response system that we need to issue and a challenge to a user. And we need to pick up something that is going to be specific to that moment in time.
However, from a usability point of view, we need to keep this very, very simple 'cause we want to make sure it's inclusive and effective. The way we do this, I proof, is we use light from the device to actually illuminate that user face with a sequence of colors and it and it changes every time. This is the way that we can meet both of those needs.
I won't go into that in any more depth right now, but if people would like to hear more about that and we're beyond that, the booth afterwards, you so please come have a chat and we'll happily go through that.
Just go talk through about the the second thing we need here, this is an arms race. The crucial thing in the arms race is about speed.
Detect, adapt, deploy the, the threat actors, they go around a cycle, they pick up these AI tools, they generate attacks, they test them, they test on any system they can biome metric, video, ident, dent, any use in social engineering attacks. They find what works. What they do is they build on this knowledge, they spread this knowledge. We see a very active information exchange in things like telegram channels, forms on the dark web right now. They then refine these attacks they go through, they pick up the new tools and they go around this cycle.
And changes in AI are making them go around this cycle faster and faster and faster. So the absolute vital thing is this needs to be counted by a defense cycle which can move faster as the the attack cycle moves faster as well. Now in the defense cycle, the crucial bit is the ability to detect and disrupt the attacks in the live environment. You need to be able to see how the attacks are evolving. You need to be able to evaluate the effectiveness of the defenses if necessary.
You need to be able to adapt the defenses and you need to deploy these everywhere and you need to be able to go around this cycle just as quickly as the attackers are going around that cycle. If you can do so, then essentially what happens is that the evolution of the attacks drive the evolution of the defense and however quickly we get this is always gonna work. On the other hand, if you can't see what's going on, you cannot possibly stop it. So an effective defense cycle, it uses attack evolution to drive defense evolution, but it does require visibility over all attacks.
The final point I'd just like to make here is that an effective defense consists of many layers with a high level of redundancy. We're dealing with a very complex threat landscape, which is changing very, very quickly. We don't know exactly how that these and attacks are going to evolve. So we cannot possibly think that a single algorithm or a piece of software can solve this problem. We need a comprehensive service that contains the many levels of defense with a high level of redundancy and crucially in here.
We need to ensure that we've got visibility over the effectiveness of each of these layers. And what this does is this gives a very strong information advantage to those building the defenses as what it enables is an assessment of the effectiveness of each layer against the attack, which is a very rich source of information. Whereas you deprive the attacker of all information except for a simple parcel fail.
So this way you can create a massive information advantage and this is one of the ways in which you can stay ahead in the arms race.
So as I say, the three things which we think are actually needed there, you need to use biometrics with strong liveness, you need to use the right form of actually liveness. You need to be able to have a defense cycle that can move forward as rapidly as the attack cycle. And you need to have a depth of defense and the visibility of the effectiveness of all those layers. And with that, oh thank you.
Does anybody have any questions here in the room?
Just raise your hand
And as usual, there is one question from the online audience, but what we've seen now does it, it essentially mean that the visual biometrics or whatever you properly call it, is essentially doomed because the AI will be faster and, but what basically respond to every challenge you would throw at it in real time. So how would you, how would you detect that?
It is a really good question and I think the crucial thing we have to think about here, AI has got much better, but we need to think about the speed of, you know, a response here.
One of the things we have to bear in mind is where do we have you a human, A human can evolve in their ability to spot things like DeepFakes very, very slowly. If you have, you know, technology in place, technology can evolve, you know, much more quickly.
But again, we need to think about the different types. If you have a piece of software that has a long sort of cycle of updates on it, it's gonna be very slow. If you think of a service where you can adapt the service very, very quickly and you can deploy everywhere very, very quickly, then the speed of, you know, a response can be very, very quick. So the essential thing is that in the technology that we have, the speed of response can match that, the speed of, you know, evolution and that's entirely possible. We just need to think about it in that context.
Okay, thank you. Question.
Hi. Thank you so much for your presentation. You were talking about the defense cycle and the rapid response needed in in the future. So I was wondering what are the technological requirements like what do you need to improve speed in that defense cycle?
So I think there are probably three things that you need there. The first thing is that you need to be able to spot, you know, evolutionary the attacks are the earliest possible point.
You know, what you have to do is to be able to understand in the wild, in the the live environment, you need to be able to see that attacks are, you know, evolving there. You can't wait until you hear about it, you know, from other sources. You can't wait until this appears in academic papers. You can't wait until a security researcher finds it. You need to find it at the earliest possible point there. But the second thing is you need to be able to understand, you know, the effectiveness of the defenses that you have, you know, immediately as well.
So, and the way that we've always approached this is to make sure that you have, you know, a lot of candidates ready to go.
So if you need to, you can test the effectiveness against all these candidates. You can bring them in very, very quickly.
The, the third requirement is you have to be able to deploy any changes, you know, everywhere in QuickTime as well. So again, this is why we see that if you are dealing with something like software where you might have, you know, an update cycle of some weeks or even months that there's, you know, it can be a limit on that. The time that you can, you respond there. If you are dealing with a cloud-based system, you can update very, very quickly, but you need to think about the ability to, to deploy everywhere as fast as you can.
Great. Well thank you. Thank you very much Andrew.
