Human-Centered AI in Automotive

Yeah, so today I'm coming from Continental and at Continental I'm working in the AI domain and bringing a lot of the human-centric approaches in the in the cars. So once you start, once you, once you trust a self-driving car with your life, you will pretty much trust AI with anything after hearing several use cases of biometrics in different walks of life, let's not leave out the automotive sector. Over the years with the massive improvements in our vehicles today, we are in midst of a revolutionary change in what it means to drive a car.

Our vehicles have become super computers on vehicles with hundreds of sensors collecting every imaginable data point, thousands of times every second. The smart vehicle of tomorrow needs to know its driver. As vehicles act as connected smart devices, technologies for authentication and identification and trust should be included in the greater transportation upgrade. Thankfully, we, there's a known solution in there to this challenge and that is strong identity enabled by biometrics.

Today I'll walk you through why is the human-centric AI necessary a few features and innovation and what the future user journey looks like. So driving is a high risk action, so the biometrics must be deployed with an eye towards safety. Contactless modalities like face and voice, work best for making sure that the drivers stay focused on the road while staying authenticated. Face recognition and eye tracking can monitor a driver's space to see if they are tired or drowsy.

It can also also be deployed as part of a driver monitoring service for insurance companies rewarding safe drivers with reduced premiums. The combination of biometric technologies and computer vision can help your car learn who you are and how you drive. Going further, the number of vehicles, vehicle theft in UK and around the world is falling, but still with quite a few in the past years.

There's still an issue to address biometric recognition for car access outside the vehicle ensures a live authorized human is unlocking the doors and starting the car, not malware, not unrecognized bad actors and inside the car built-in sensors can identify and authenticate drivers and passengers alike. We have worked on several of these technologies, face recognition, anti spoofing, et cetera, and over then you. And combining all of these, we have also come up with biometrics two factor authentication.

With occupant localization, you are able to identify who is occupant and where is ccd inside the car. And then you have speaker separation to separate the audio quality, to separate the audio of different occupants in that, in those particular zones, which also helps in conversation with the respective a i chatbots as well. And also with focus of attention, we are predicting where the driver is looking at, at a particular moment. Is he attentive towards that road or is or is or take attention somewhere else.

Next, with the drowsy driver drowsiness detection, we use eye landmarks is I as aspect ratios detecting to detect the degree of eyes open and we class, this is classified between into alert or slightly drowsy, drowsy or sleepy. On the other hand, with attentiveness detection, it combines, combines a few more approaches. Here you have head pose estimation, gaze estimation, and this is used to calculate the duration of glands for how long are you looking somewhere else out of the in, out of the necessary gaze area.

And the degree of warning depends on how on the vehicle speed, if you are driving at quite high speed and also gazing outside are, are you looking somewhere else for a long period of time? So then that is much more distracted and the emergent, the alarm will alert will come earlier. Now let me take you towards the user journey with here we have a device list access this, this was presented first in this year at CES and here basically you are able to walk towards the car and access it without using your key fob, without using your keys or even your smartphone.

As you approach towards the card when you are around one meter from the card, the face recognition will be activated. It also uses radar to sense that they, someone wants to access this car and the the, the display that you see in the B pillar of the car comes alive only when only when someone wants to activate the face recognition and want to, wants to authenticate. Over here we have partnered with tris and, and their, and they and we are also using their proprietary 3D dot projector or anti spoofing system.

So we are really detecting the quality, the skin of the person and ensuring that there is no high quality mask, silicone mask or anything being used to spoof the system. After that, as you are getting into the cabin, there are several different parameters that we, your parameters or services that we offer.

We are, we would be calculating stress or discomfort of the driver drowsiness and what, is there any objects that at that's the occupant is holding, whether it is hot C hot, hot coffee or a mobile phone or something else that would be distracting in cabin payments are enabled and if, if there are any specific restrictions like child control or something like that or even if there are specific personalization preferences, that is also taken into consideration health and wellbeing is insured.

And also as I mentioned earlier, distraction is also a calculated when you are inside the car, in case in case there is a braking break in of to the, to the vehicle and there was an attempt to bypass the face face recognition. Then inside, once you are inside the car, there is another level of biometric security and this time it is to secure the any engine start. So only once you are authenticated can you start the engine.

So yeah, and once you are, once the occupant is recognized, be it sitting at whichever position, the driver or the the poor rider or the backseat people. So the seat and the cabin can be personalized accordingly. Seat positioning, height, height and distance, et cetera. And then you can take the AC system as well. So all of this can be personalized. Some of the cars, some of the high-end cars also have customizable high support and the lumber support. So even those things can be customized. Then next you have the infotainment personalization.

So your profile into YouTube music or Spotify or something else can be automatically logged into so that someone else's feed is not polluting your feed and everyone can enjoy their own stuff of liking. Also with this, the conversational ai or you can say the empathic companion can also learn who you are and accordingly chat with you in a way that is more familiar and comforting to you. Next you have conversational commerce. So since we are, since we are able to con continuously authenticate you by using face or voice, the, yeah, the payments can also be enabled.

And next, then you have personal assistant over also, which has access to several other modalities as well. So you can access different apps, you can access third party assistance or you can, you can also au automate some things at your home app, use them remotely, use your vehicle features, et cetera. And also when you combine, you know, with health rate, health rate and other health data to measure stress and alerts. So ultimately over a period of time we can see if you're not feeling well, then we can alert the nearby hospitals or nearby or or or your family members also.

And if your car is autonomous then put it on autopilot also for fleet, for fleet operators like Uber, et cetera, we can, you can evaluate whether, whether the right quality is good or not.

Today's, today the ratings are subjective, but with such evaluation it can be, it can be objective and you know, better grievance literacy can be made as well towards safety based on who is sitting where and the age and gender of those people by evaluating the body height and mass and detecting the presence of child, et cetera, how you are seated and what kind of objects you are holding or what activity you are doing. Accordingly, decision like firing or deployment of airbag can be taken whether or not to deploy airbag.

And if, if the airbag has to be deployed, then with how much force does it have to apply because if it is an elderly person or someone, then you might actually end up injuring the per the, the person because of higher force of this deployment and not instead save them. So that is again, a little bit dangerous over there. Child presence is also detected and if if the, if you, if the child is left behind, we without adults sometimes in case of some shopping or something. So then also the the, the parent is alerted some by using a phone by, by using a phone or some kind of alarm as well.

So, and this is and functional demo that we have of the occupant safety monitor. And here you see that if the person is leaning back or leaning front, then that kind of signal is activated on the left side. And in this we have face recognition as well.

So you, and after face recognition, we are also automatically classifying the person as as adult, I mean based on the age, gender and also that. And when you see that the child is there. So the child was also directed and that particular mark over here was highlighted. Now he is holding a phone. So that object comes up. And this is important because when, when you are driving, when when you are not driving in case of you know, newer cars and the car wants to give back the control to the driver.

So you need you, it needs to be from, it needs to confirm that the driver is ready to take back control in some difficult scenarios and also if there's a coffee or something. So you, if the airbag is deployed, then again the hot coffee or hot water or something can, that can also damage the person. So what's next in the biometrics using one, using biometrics as one factor or two factor four, vehicle entry ignition, et cetera.

It'll help in making our vehicle secure and identified driver can potentially lower the insurance premium thanks to verified safe driving low glucose levels can cause people with diabetes driving car to lose consciousness. The occupant health monitoring with vitals will make traveling safer by monitoring sudden temperature changes, fallen levels, pollution in the air triggering asthma attacks right ex as we move towards level five automation, the OEMs, the car companies will focus on improving the ride experience and the dynamics within the vehicle.

Therefore, effective computing will be quite essential and to understand and monitor and monitor occupant behavior. One of the major drawback of deep CNN's for use in safety critical applications is their black box nature and explainable AI will go a long mile over here in ensuring that the decisions taken by these AI models are validated and auto audited as well. Fairness in, in made in multiple studies and now it is quite well known that the AI models, they they tend to have some biases et cetera.

And it is important that this is also again taken into, into consideration when put into cars as well and we don't end up, you know, injuring someone or anything like that. And lastly, drunk driving technology that uses sensors, cameras, and AI is upcoming and will be integrated in vehicles. It is a challenge challenging space with the rise of, with the risk of false positives and the reality of drunk drivers who will be trying to feed the system. So it's going to be an interesting problem to solve with this I and my presentation. And thank you. Thank you Shahan.

So just one question yesterday I asked whether the training of ai, large language models in English, whether that's intrinsically unethical because it doesn't train in other languages. As an New Yorker, I can attest that there are different driving styles. New Yorkers tend to be aggressive drivers. Will there be the ability to also take into account cultural differences in driving styles as part of the modifications and customizations localizations for these systems?

Yeah, I, I think that will surely be the case all across the world. I mean there are different styles. We actually also are working on identifying driving styles, driving profile as well. And you know, this will, this will form a key input in how the autonomous driving is also happening.

So, and, and how the other vehicles act according to this. So I mean if you are, if someone is an aggressive driver, the car, since the cars will also be communicating within themselves. So this kind of communication can also happen about the driving style of this particular car, what is happening over here. And accordingly some of the nearby cars or something can be, can be alerted and thereby, you know, take necessary action or maybe like, maybe stay slightly, maintain some distance or something like that.

Also, like, I mean on, on the eastern side of the world as well, I mean there are smaller, smaller sizer size of vehicles also there are more two wheelers, there are three wheelers as well, you know, top tops in Thailand, et cetera. So, and they have, you know, a high degree of freedom as well, like the front, there's only one tire in front so you can really move along very freely. So these are quite new things that, that the autonomous vehicles are yet to see. Fantastic. Well please join me in, thank you Shashank, for a very interesting and provocative presentation.