Successful and Unsuccessful Approaches to API Security at Ford Motor Company

So my role at Ford, I said, among other things is I'm, I'm in a part of our vehicle, cyber security. So think of our products, you know, the vehicles that, that you as a consumer would drive or purchase and, and, and interact with, and APIs are a big part of our, our products as we go forward. Let me go to the next slide here. So any incorporation, any large company, small company, either I call it what I call the cybersecurity snowball effect today there, the proliferation of technologies, lots of new developers, open source, agile methodology, the cloud, all these things come, come into play.

And ultimately at some point in time, you know, you gotta hit a point of no return where you really just can't keep up with it without an approach. So what happens ultimately in many large companies or small companies, eventually you get too late. You'll never outrun that snowball fast enough. It's gonna catch up and you'll have a security breach, which is not good for any company, any organization, our consumers, our products, it, it, it just a, a bad thing to happen. So how do we try to prevent, you know, this security beach and, you know, being run over by the snowball.

So in a large enterprise, we're a global multinational company and we ask these fundamental questions, failure modes, okay. What are the failure modes in a global enterprise around API security consequences? And more importantly, how could we remediate these cuz at the end of the day, you know, we have to be successful every time that the hacker only has to be successful once you're to cause a security breach. So what are we trying to do, you know, to be successful and put this defense in depth approach in place. I'm gonna tell you what, what doesn't work. Okay.

So first I'm gonna talk about different failures of approaches and the one very common approach that I've seen many organization Jews is I call the whackamole approach, the cybersecurity. If you've ever played the game in a, in a carnival or snowballs, eventually, you know, the moles, you whack, 'em all with a hammer and you knock it down. But eventually the MOS come up too fast and you won't have enough hammers. And eventually some of them are get through, this is the same challenge you're gonna have with, with cybersecurity.

You know, you have all the bad people, the bad actors on the right, you know, whether it be security research or terrorism, service mischief, you have your developers, you have your apps, whatnot. And ultimately if your approach is when we find something, we'll stamp it out, it is an approach. But ultimately, you know that snowball's gonna get you and, and, and you will result in, in bad things happening to you. So a large company, it's great.

It's like, well, we have standards, okay. Standards are great. We have API security standards. We have style guides and you know, you can try and approach the, to mandate policies and standards for everything. Okay?

However, that's not enough. Okay. Reason being is, you know, we have thousands of developers at Ford motor company. You can imagine at any time new people coming, people going and be realistic. A policy is only effective when they understand it. And they actually use it as part of the development process. And it's automated into the process. So standard is, is good, but again, it's not enough. So next thing is, is a command and control approach. I call the security.

Okay, very hierarchical. You know, I'm the boss.

I said, so, okay. It doesn't matter what your teams want. You just do it. We have autonomous product teams and that's part of agile. You be able to do that where they, they have as part of this approach to shift left and DevOps insect DevOps, where they're accountable for it. So command and control doesn't necessarily work when, when you get into a more agile or, or product team focus approach to that again. So this is key.

You must listen to the feedback from these teams, because if, if you just dictate policy, without listening to them, you could actually be harming potential changes to your products, you know, things in the market improvements and whatnot. So it it's always as collaboration with between cybersecurity as well as the, the product teams are actually delivering the products and services. Fear. Okay. Very much like that. You can drive fear. Fear is a great motivator. Okay. But it's a very short term motivator and it's a negative voter. Yeah. So emotional outburst, it's like, yeah, do it.

I make the policies and you follow 'em and you'll do it, or I'll fire you. Okay.

You, you can say to a point that actually worked for Amazon. If you look back the, the Amazon jet Bezos mandate that says, you know, you will do APIs or you'll be fired. Thank you. Have a nice day.

You know, you can say that it worked for Amazon. It was they're very successful. They have a very, very large API portfolio today. Very well done. But at what cost does that come in, nothing makes individuals more uncomfortable than fear. So use that when, when you need to.

But again, as a motivator, it's a powerful, but it's a very destructive motivator. If, if you'd not used properly Governance, we're an organization run around everything through governance. Okay. Checkpoints are good. Okay. You want to have any point, you know, through, through a process to make sure that you're matching you're meeting certain quality objectives, you're meeting security objectives or whatnot. But if you're working in an agile and iterative man iterative manner, it can often be incompatible with that.

If I have a governance team that we meet twice a week for one hour, and you can't do anything until the governance approves it, and I'm an agile team, they, they kind of clash. So this is a case where I'm gonna call agile government governance and automation are really key to help, to able to do it again. The concept of trust, but verify is very, very important. We do that in many of the things that we do, but it's automated.

It's, you know, you codify the rules and you let the automation and the tools do the work for you. If you do these things, I told you all these failure modes, here's what you're gonna get. And that you'll usually get ignored. You'll come, or you'll get, I call malicious obedience. And you'll hear this thing coming from the product team saying, Hey, our product is late because of these cybersecurity requirements. So you you'll become very quickly the organization that says, well, you're the reason I'm late. Not necessarily cuz of bad practice or whatnot.

So it's definitely where you don't wanna get into. If you see this, the organization, it's a sign of, you know, not maturity around your APIs and people at the end of the day, you know, people are familiar with firewalls are, well, guess what product teams are gonna protect themselves. They'll put a firewall around themselves. Okay. So a cybersecurity professional says, Hey, Hey team, you have to remediate this because our tool or checker, our thing found this. It's like, they'll put this firewall up in front of you.

The developers they'll give you someone and say, well work everything through Joe or Mary or Sally. Okay. Why? Because they're resistant to company creating an app security program for fair being slowed down and delivering their products. So this balance between delivery and security is, is, is key.

Boom, which goes to it's a balancing act. When you approach anything in, in cybersecurity, especially for APIs, it's a combination of three things. Always people, people do the work processes and technology automating those processes to, to, to power the people, to do the work and to do work quickly. And I always remind people that it's cybersecurity is not a, a security problem. It's for us as a business, a shared business responsibility of people processing technologies that work together to manage risk. It's that simple.

I mean, if you don't manage risk, the bad guys are out there in this case, the snowmans are out there and they will get you. It gets just a question of how long and how quickly they will get you. If there's one key message to all of this is actively listening. Okay.

You know, the great quote I have on there from Bob Lewis on the left hand side is relationships proceed processes and outlive transactions. Getting to know your teams, getting to know what they're working on, the products, the difficulties in all they're having. Cause the end of the day, a corporation such as Ford or any company, it's really a collection of relationships and with good relationships, everything can work and it can work really well without them. Nothing can. So poor relationships can sabotage many, many things, especially these initiatives.

If you're trying to implement a cyber program for app security around APIs. So what have we done at Ford? What are some of our approaches that, that we're doing? The reality is roll up your sleeves and become part of the solution. Okay.

There's a, there's a, there's a term I used developers. Don't appreciate unfunded mandates. If you in cybersecurity says you need to do these 10 things.

You know, I'm not gonna help you. You know, they're already overloaded. They're already burdened, you know, and you're not helping them. So automation is key. We must find ways to eliminate manual work whenever possible and augment manual efforts with automation in governance, in the release process, in the pipelines, in the delivery of it only then will you, will you have a very good API security program because otherwise you'll follow yourself into the whackamole problem. Like I put up earlier and at the end of day, it's empathy again, this is about relationships.

You know, empathy is a skill as a cybersecurity professional, as a software engineer is sharing someone else's feelings and it, and it really separates us from, from, you know, technology technology will work automation, great technology works. It doesn't care. It doesn't have feeling, but people do. At the end of the day, we have developers, they have dreams. They have good days. They have bad days. All these things come into play and you need to really take that into account.

And at the end of the day, I call that that's our secret sauce for security as human interactions are at the heart of almost every issue. If you talk to any agile coach, they're not gonna tell you the problem is with agile it's with relationships between the people, the methodology works.

It's, you know, these nasty things called people who, you know, have trouble relating to one another. And this is really, and cybersecurity is no different. This is really one of the most important skills you can have. So what is our strength?

You know, how we strengthen our API security posture at Ford. It really, again, it five simple steps, you know, understand your inventory may sound simple. You can't manage what you don't know. Okay. We have thousands and thousands of APIs at Ford. So you can imagine in any point in time, any of those changing, having control of that inventory is key. Knowing when they change, audit your sensitive APIs, we all have to follow compliance. Whether it's GPDR, whether it's art and our case, we're doing vehicles, they do command and control. We have cybersecurity laws.

We have to follow from U N E C E and other organizations. Again, you have to know your sensitive APIs and attack 'em and audit them and understand how they work attack the O ops top 10 security, you know, APIs. I mean now have their own top 10 list of, of OWAS problems. There are a unique set of, of technology that have their own top 10. There's plenty of tools out there that can help you attack those top 10, both from a shift left from source code, all the way to the monitoring side of it.

Trust with verify again, key with that, trust your teams, but verify through automation and compliance and then go to the outside. Don't think that, you know everything. So in our case, we, we have coordinated disclosure and bug bounty programs engage the outside, the outside, outside world. They will help you. They'll help you a lot, you know, to, to find vulnerabilities in, in your products. So we that's part of things we do so can very simple, but you know, there's a lot behind each of these things. So in a nutshell, that's kind of how we got to report it very quick.

And I'm gonna hand over now to Isabella, it can tell you how we were using tools such as 42 crunch to, to deliver on this vision that I I've spelled out to you. So, Isabel, Wonderful. Thank you very much, Darren. Should I share, or do you want to continue to share yourself? I'll continue. I'll continue. Fantastic. Yeah. So you know what? We're listen to, to Darren and explain all those great approaches that are taken at Ford. One of the things we've seen with them and, and other customers working with is what really pays us to be much more proactive about security.

What you want to do is ensure that your APIs are as secure as possible before you actually get all the way to production. You went all the way to that API life cycle, right? And the reason for that, the next slide is really because you will have a design time to take a lot of decisions, right?

And, and you have this great cartoon here about the design and some decisions you think of building an API as defining the blueprint of what your application is going to be right. You, before you build a house, you have to do a blueprint and then you have to use that blueprint as the guideline to developing your applications in your code. So whatever you don't design and think first could be really, really hard to fix after effect.

And, and there's tons of examples of that. It could be, how are you going to, you know, do the, the authentication and authorization on your APIs. It could be how you're going to design the data that is being exchanged through those APIs. Darren was talking about sensitive APIs.

Like, what are your APIs? Are there sensitive? What kind of data are there actually transporting? And based on all those different factors, you have to take decisions that can be really, really hard to change once you reach a certain point in the API life cycle, right? So that is why we, you know, really align here on the fact that we have to start by defining what is the contract? What is our API doing? If you wanna go to the next slide? So what is a contract, right?

A contract as in any contract is a commitment being passed between the consumers, the producers of an API and the consumers, right? So not only is this going to impact actually security, it's also going to impact a lot, the quality of the service that you're going to provide as a service provider to your consumers. And this is true. Should you expose your APIs internally? Or should you expose your APIs externally? It doesn't really matter. In any case, you should really consider the people or consuming your APIs as your customers, right?

And, and as your customers, you want to offer the best service and the, and that you can to them. So that contract is going to help you define, Hey, this is what my API does. That's how it works. This is the data I'm going to expose to you. Those are the operations I can call. And this is how you're going to call that now, how do we do this?

Well, we're lucky enough that in the API world, we have standards to actually do that. If you wanna go to the next slide.

And, and one of those key standards is open API. You may well have heard about AC and KPI. It's really the same. I would say goals that that open API had when, when, when got started, which is to have a single standard way across all kind of products and all kind of functionality to describe, Hey, this is my API. That's what it does. This is the data. That's the security, et cetera, which allows us then to speak a common language.

And this is very critical to what, you know, Darren just explained, which is how do we get all the different people that are, have to collaborate, you know, back also to one of his point, which is to say security is not only of one consistency and just the security team. It, it is really a team job. And it's across multiple people, the dev people, the ops people and the security people. And if you want this to work, then all those people who are involved have to kind of speak the same language.

So that is the goal to go to the next slide that we're taking with, with the open API, with a contract. So once you have defined a contract, what are you designing?

You say, well, this is, this is what I'm promising to you. I'm promising my API is gonna work like this for you.

And then, well, we have to do as long the life cycle to a validate, this is actually what we're doing. Are we respecting the contract as the provider of that?

And then, then force that, you know, at run time, this is what the API actually is going to do, right? And, and again, this serves really as a shared understanding across all the different teams and more than that. So I'll just add the point to this, which, which we see at all our customers is the key advantage of using something like open API or C, is it a static description? Is it tile? It's a text it's Yamo or Jason, right? But it's basically, it's a test description of what your API is doing.

And whenever you have something written as text or as, as, as more a programming language like Jason LA, Yamo what allow this allows you to do is to version is to make differences is to end rich automatically you, you have a lot of different capabilities which are offered to you beyond, you know, again, the security, you know, if you've never worked with open API before, one of the key things this allows you to do as well is to take an API design first approach by which you first define, Hey, this is my contract.

And then while the developers will go and implement the backend, and during that time, people are gonna consume those APIs can start working on the applications. You can do automated testing, you can do automated generation of mocks. There's a great, you know, number of advantages beyond security by just speaking, everyone speaking the same language, right? So on next slide from a security standpoint now, so this is kind of generate advantage, but from a security standpoint, what also an open API based contract gives us is a positive security model. So sure.

There's plenty of practitioners here that I've heard about this term. But the key thing is traditionally from a security standpoint, because we didn't have a way to read, describe what is a traffic coming to us? We've been turning to a negative security T model by which we look at the traffic. We have a set of signatures of patterns, of the things we don't want to see, or we think represent a threat, look like injections or scripting, et cetera. And if we see those patterns, then we'll just block those, those injections and, and those potential threats, right?

So that's a negative model, which basically means every time there are new threats and new problems, we have to go and update those signatures, update those definitions of those patterns. Otherwise, well, obviously we don't know, they go through because we don't know how to recognize them, right? So it's always been kind of a sticker ground to, to turn this into a positive security model by which I know exactly where the traffic is going to be, which allows me to say, well, whatever matches, that's cool. I'll let it go in. I know this is my known API and it works the way it should be.

And anything else that I don't know about it could be bots. It could be past and rehearsals. It could be bad tokens. It could be specific API issues like having content and payloads, which you do not match what we're actually expecting. And by the way, this is true for APIs. This is very critical to understand for, from traditional security. It is as important in APIs to validate the response. And it is to validate the requests because you wanna make sure you don't evade data that should not be evading. So this one contract is gonna also serve that purpose. So how do we use that at 42 crunch?

And this is the next slide. We have a set of tools that, that Ford is using today to a, you know, know, again, what is the contract? Is it good enough?

Does it, you know, follow all the corporate rules and requirements that we wanted to enforce those rules that we want to enforce. We make it very easy for developers and everyone to actually enforce. That's the first thing we do. And the second thing is do this testing.

It's like, okay, you told me, this is how your API is working. Let's go and test it now. And that's the scanned implementation of that. And the last part is, okay, now we know how my API is working. We validated the contract. Now we can do automatic protection in place. And what is really critical. And this is on the next slide is the automation part of all this. And Dar has insisted this many times. And we see this at all our customers as well.

Again, they have hundred thousands of APIs. We have other customers.

It's, it's what we see at all our customers. You don't have 2, 3, 5 APIs. You have ver you have thousands. So it just doesn't scale at all to go and do all this validation manually. It just will not work.

What you want is, is put in place this virtual circle, by which every time an EPA is created or is being changed, you have this automated pipeline that will go all those, do those validations and force this compliance against the rules that have been established by security, and then automatically thanks to the approach of taking a, a text based definition and, and description of my contracts. I can automatically inject some of the policies I want to enforce, whatever it is. It could be rate limiting could be JWT, validation, token, valuation.

It could be injecting specific errors automatically, whatever it is that you want to do from a security perspective. And then you can automatically deploy that against a runtime that automatically interprets and engages those policy, right? So that you have that continuity. You have that automation and, and it happens very easily. Every time developers actually change something, right? So you enable that virtual loop by which you do have control. You do trust and verify, but you also have this automation, which makes that is not a burden on the security teams.

And it becomes basically part of their everyday life. Right? So as a conclusion, we would say, I don't know if Darren, you want to go and, and take a conclusion. Sure. So this is basically, you know what we we're doing it for today. So contracts are, are seen as commitment.

So as, as a producer service API, that's my commitment, both from a quality and a security aspect to our customers, whether it's an internal customer or whether it's you, it was a consumer, whatever. So that's really key on that. Focusing on that second is intentional design. You can't do intentional design without actually having a design.

You know, I've seen plenty of 'em with where we, we ended up with an API and it's like, you look at, it's like, like, what were you thinking? You know, and, and trying to retrofit that, it, it becomes very, very difficult.

Again, customer experience. Again, this is key.

It's a, if at the end of the day, I would ask, I, I, I challenge our developers. Why do we treat each other so terribly? You know, why don't we provide good contracts to ourselves, you know, to have a good customer experience cause poor customers for our developers. And if it's not defined, you know, the hackers love it, vulnerabilities know very easily the vulnerabilities get in because it's not well defined automation.

Again, we talked enough about that, but don't do it without automation. Don't even think this will work manually or scale.

So, you know, go back to that snow plow effect, you know, the, you know, snowblower give the tools to your development, put it in the process and make it work. And it may sound trivial, but the shared language already call the single source of truth. If you don't have that, your operations team, your security team, your software engineers, they'll be talking at each other instead of talking to each other. So getting this shared language is shared source of truth.

You know, just like drawings do in, in, in, in construction. You know, this is a source of truth for building whatever the API contract becomes. That for APIs, it's a really powerful model. If you haven't thought about it that way.

I, I challenge you to think about it because it really can drive your business very, very well and, and move you along this API maturity very, very quickly.