I want to speak to you today about cybersecurity programs and our journey in establishing what we feel is a comprehensive one. And some of the learnings that we've had along the way. I want to start by explaining my long title. Then I want to talk a little bit about our business because it's, it's a very specific use case that I want to describe. And then I want to get into the meat of it. Talk about the things that we've done. I want to talk about automation, and then maybe we have time for questions at the end. So accelerating cybersecurity, the age of acceleration is, is not my term.
It was coined by Thomas Friedman in a book some years ago. And it kind of, it's not a cybersecurity term.
I, I want to put this in a broader context. It describes a set of criteria that we see in today's world, right? And that I think applied to cyber security, not gonna read them out, but this is the environment that we find ourselves in, right? So effects are no longer linear. Our systems are becoming increasingly complex. And obviously, you know, we yesterday in the, in the workshop, we talked about known unknowns, unknown unknowns in terms of threat, right? This is
Kind of the uncertainty principle. And of course you can under also understand acceleration in a very kind of literal way. Right?
If I look at the number of reported CVEs over the years or the number of data breaches over the years, I mean, I'm not gonna, I just want you to appreciate the way the graph progresses. Yeah. Or the amount of monetary damage.
And I, I believe I see a trend there that is, may be related to an acceleration. So that's where my title comes from. And I want to talk about, you know, how our business is responding to this and maybe in a broader context, what, what your business could learn from this. So who are we?
I mean, obviously, probably most of you have heard about eBay, but EBA classifies through is a slightly different beast, right? So we're not the auction site. We are a conglomerate of 10 brands across the world that have a similar business model.
I mean, probably if you are from Germany, right. You've heard of KHA. If you're from the UK, you've probably heard of Gumtree. All the others have a similar business model. So we are a content provider. There's some statistics there, right. We have a large code base
And that's literally all we have, right.
So we put, we write code and then we put it on the internet. That's what we do. And so our security challenges are mostly related to software security, life cycle, right? Secure software development. And if you remember, you know, eBay already, in terms of cybersecurity, eBay already had its wake up call. There was an event in 2014, that's been publicized. I'm not gonna go into many details. Right. That really prompted a change that made us go from this collection of 10 startups yeah. To a corporate approach to cybersecurity.
So in terms of technical, operational, managerial, and policy development, and, but at the same time also, and this turns out to be our greatest challenge, right. In terms of mindset.
So in, so in 2014, we really doubled down on the cybersecurity effort already. Right. And that's been a few years ago now. So what have you learned?
Well, we've, we've amassed this large portfolio of data of about the threats that these 10 brands and these 30 million lines of code face. Right? So we have some ticketing system that has data about the kind of threat when it occurred, what people worked on, the remediation who reported it, how often these things happen. And of course, on the other hand side, and that's what I said earlier, what we have is our code, right? And there also we have
Data about who commits this code, what teams work on this, how often this code gets changed, how complex the code is.
And the, what we did earlier this year for the first time is right. We looked at this data and we analyzed it. I was nearly gonna say, we put it through some AI, but I'm gonna prefer to call it complex Matthias instead, because of what we heard earlier.
And so we, we drew some learnings from this and you know, there may be not, they're not the great revelation, but at least now we have some data based. So some founded evidence of what goes on with this, in our software security development, right?
I say, none of this is super surprising thing. The kids principle still applies the larger your team. The more brightly people think about how to write secure code, right? So a single person working on code is a higher risk. If you change often your so high modification frequency reduces the risk of introducing some vulnerabilities. And the interesting thing is, or this, this was kind of the most surprising learning to us code that got worked on by people who have since left the company
Has a higher risk of having vulnerabilities. Right?
So that might either mean, I mean, we haven't really concluded what that means either. It means all the bad coders leave or yeah, we don't really know, but especially the last point, right? The high modification frequency. So we pride ourselves on being agile. Right? All of these 10 companies work in an agile software development methodology. We have continuous releases, right? So we change our code on these websites 20 times a day. Every time a new feature is rolled out, it's immediately put life, right?
The, the QA processes follow the C I C D model. And this is somehow working for us. Right? So the data shows that this is somehow working for us, even though we don't really have a clear understanding what it is we're doing, right.
But I'll, I'll come onto this because this, the traditional paradigm right. Is no longer working. So shifting security to the left for us means everybody is now responsible for security, right?
This it's, it's essentially a problem of the developer, the Amazon CTO already recognized this last year as well. Right.
And, and our experience really echoes that.
So in the traditional way, right, you look, we looked, or we used to look at our code in production, right? We run pen tests against it. We have some security scanners in our network. We even run book Bonti programs, which incidentally, I'm a, I'm a great fan of, but this, you know, when you release 20 times a day, then an annual pen test makes no sense at all. Right. To give you an idea of the quality of, of your security posture. So then about two years ago, right, we started to really attack the, the code, right?
So automatic scanning tools, dependency check, secure design guidelines. And for example, without going into too much detail, the, the largest majority of, of vulnerabilities that we see in our code is cross site scripting, right? And you would think that with all the template engines that we mandate and after all, you know, it's 2018 cross site scripting in websites should be a thing of the past. And while we found out that actually we are doing most of the things, right. Our developers doing most of the things, right?
The, the vast majority of those vulnerabilities come from
External open source libraries that we include in our code. Right? So I cannot stress enough how important dependency checking is for your secure software development. And now this, this year for the first time, you know, we went a step further and this is based on the analysis that we just did, right? So we've always done. And I'm sure everybody is aware of the importance of security training. We've done this for a number of years. We're raising awareness.
We have online training, we have onsite training, every new joiner to the company, get some security training. So it's a mindset challenge. And it goes with ownership, right? This is what I said about developers, leaving the company. We still find that we have code in production that hasn't been touched in a few, let's be optimistic, say in a few months, right? Not to say in a few years, this is no longer. This code is no longer being developed. It's not owned by any particular team because there were two reorgs of the company structure in between.
But this happens to be our authentication system, right? For the, for the website.
Well, how you know, so
These things remain a challenge for us. It's a mindset problem. And on the other hand side, of course, you know, we are going with the technological advances. So we started out when the world was our cybersecurity world was understandable. I read bare metal service. Every asset had an IP address. We used to audit the puppet code and we had some super firewalls, right. To provide segmentation between these different businesses. Then a few years ago, we moved to an AIS system, right?
So now we find that there is already an existing tooling gap opening where the, where the security scanning tools can't keep up with these changes. Right?
So, I mean, I'm not sure this is maybe too technical, but so we deployed our own internal open stack cloud in, in cloud IP addresses don't matter anymore. So if you're scanning tool works on like peer addresses, you need to upgrade auditing of security groups. That's the network level segmentation in open stock is more challenging than with traditional hardware firewalls. And then of course, you know, we moved to the next level because that's what our developers want. Right? We're now all on DOCA. Everybody has DOCA and Kubernetes application layer, walls, okay.
Name, space isolation. You know, so the entire infrastructure is converged isolation, only on name space. You don't actually know how many assets you have anymore, right?
So you, you can't, you can't even compile an entire asset list because the number of your web server instances changes on demand by the hour. And so these are different challenges that we are addressing and where we are violating the kiss principle, because now we run all of these three things in parallel, right? So we are very bad at switching things off. And as we're progressing, the security team doesn't grow, but the requirement list grows steadily. So these are just some of the insights from our journey, right? I'm sure this might re resonate with at least some of you.
I say it's, it's a very constricted use case. It doesn't cover the wide variety of cybersecurity challenges that you might have, but what's becoming clear, not just from our experience, but also in general terms, right in agile, continuous development. There's the, the vast majority of proponents still stayed a lack of automated, integrated security testing tools. This is research from earlier this year.
And again, what you see there is, you know, many of, a lot of this apart from the tooling thing
Can be, can be addressed with, or is essentially a mindset problem, right? It security slows me down. The tool reports too many false positive. I'm just not gonna bother with it. Right. And so in summary, I just want to make four points. And then I open up progressions. So the threats evolve quickly, the technology evolves quickly in cybersecurity. Your program might not evolve as quickly. That's at least our experience.
And we have a continuous challenge to adapt good tooling and good automation are key. And we've seen many, or we will see many talks today that speak to the same principle, the acceleration gap, the uncertainty principle that, that I was referring to earlier. Right.
I, I do believe AI is a great enabler in that space. And overall it's the people, it's the mindset challenge, right?
There is, there is a reason why AWS buckets end up open on the internet, right. That's because somebody did it wrong and you, you know, so you have to be aware of these things. And that's me done for today. Thank you very much.