Unlocking the Potential of In-Vehicle Edge AI

0:00 Introduction to Automotive Innovation

Welcome to another episode of Driving Innovation, the podcast that explores the intersection of automotive software innovation and AI and how it’s driving the future of mobility.

Today, we’re diving into how software-defined vehicle technologies and solutions can enable edge AI to transform vehicles from static machines into intelligent, adaptive platforms, a topic we cover in our latest white paper, Unlocking the Potential of In-Vehicle Edge AI that you can download from our website.

0:31 Challenges of Cloud-Based AI

We’ll break down why cloud-based AI isn’t enough, what’s holding OEMs back, and how SDVs unlock real-time in vehicle intelligence, from predictive maintenance to energy optimization and beyond.

Whether you’re a business leader shaping your company’s digital roadmap or a technologist driving changes under the hood, this one’s for you. Let’s get started.

1:04 Guest Introduction

With me today to discuss in-vehicle edge AI is Steve Stoddard, product manager here at Sonatus. Steve, welcome to the podcast. Thank you, Sanjay. Well, let’s get started.

What is the buzz around edge AI in the automotive industry today?

Yeah. I think there’s a number of trends that are kind of converging right now. One of which is autonomy that has been developed over a number of years here over the last decade or so, and it’s finally starting to really come to fruition. And so there’s a lot of buzz around that.

But also, you have the confluence of these traditional LLM models that are really expanding quickly. And while they’re mostly based in the cloud, it’s really bringing a lot of awareness to AI and bringing a lot of customer interest and actually expectation around having AI in their vehicles today. One of the issues is that cloud-based AI has a kind of a limit because of the data costs associated with sending data from the vehicle to the cloud. And there’s always potential connectivity issues.

And let alone there’s also the concerns around PII data, basically personally identifiable information or possible data leaks from customer proprietary information.

Things like GDPR in the EU and other regional-based regulations that would require a little bit more sensitivity around control of personal data, all lend towards wanting to run AI models in the vehicle as opposed to in the cloud.

2:24 Moving from Cloud-Based to Edge AI

I know AI is all the rage these days.

What is happening in the automotive industry today to make edge AI more possible that perhaps wasn’t there before in some of the more traditional legacy vehicle architectures?

Yeah. Definitely. Software-defined vehicles, the sort of SDV trend definitely is a big factor in that. Of course, when things are more controlled via software where you can change certain variables, then it enables a lot of capability for AI models that maybe didn’t exist before.

Additionally, sort of a lot of these ML models have really become more established. And so traditional ML has kind of paved the way for things like recommendation models and things that have been used in the cloud today, which now are starting to be better optimized for running on limited hardware that you see in the vehicle. At the same time, some of the hardware that’s being developed in the vehicle, there’s more of a trend towards HPCs and compute capability that may be better suited for AI neural compute and GPUs, especially with the rise of autonomy as I mentioned. There’s a lot more focus of these types of silicon that can be implemented into the vehicle.

And so there’s a lot wider variety of capable ECUs in the vehicle today that actually can run these types of models. So clearly SDV technologies opens the way to bring edge AI into vehicles.

Let’s dig into that a little bit more.

3:52 Barriers for OEMs in Edge AI Adoption

What’s been holding OEMs back up until now?

One of the big things is going to be the network topology in the vehicle. So the actual architecture of the vehicle and the signals traditionally have been more rigid and static based. As we move to SDVs, of course, that flexibility introduces a lot more of this capability.

Another problem is the actual data access. So one of the things that one has to think about when deploying AI models into the vehicle is how do I send the right data that’s required as model inputs to that model at the right time.

And that’s one of the things that certainly the newer technologies are starting to enable. Additionally, there’s a lot of cloud-based and sort of MLOps types of tools out there. Very good for deploying models in the cloud, but that’s the more traditional way of doing ML. There’s less fewer resources available, fewer tools for deploying models into embedded devices like vehicles. And I think vehicles have a little bit more sensitivity requirements compared to maybe just general IoT devices, particularly for the data control access and things like that.

4:58 Automotive-Specific AI Tools

Right. So flexibility and precision, obviously, needed for running AI models. What you mentioned about more automotive specific AIML ops tool chain. Let’s explore that a little bit more. Can you elaborate on why all of the tools that are available out there may not be suited for an automotive environment?

Yeah. Definitely. This is this is a tricky one because the folks who have access to the data coming from vehicles, typically the pathway for training and deploying a model would involve gathering all the data, building the model from that, and typically that’s going to be data scientists or ML engineers.

And then ultimately, to bring it into a vehicle, you have to make sure you’re connecting it to the right data sources and optimizing the model, typically downsizing it, implementing certain techniques like sparsification, then designating where you want to deploy it into the vehicle, and ultimately the approval process for saying, I’m good with this model. I know it’s not going to harm any other processes in the vehicle, and I can go ahead and actually deploy it. So we find that ML tool chains really need to hit upon all of these key facets. Otherwise, it’s just a roadblock. You won’t be able to execute.

Right. Yeah. It’s not like deploying AI on a big server. And plus it’s a device or a platform that has to do a lot of other things. Primarily, you know, transporting people goods from point A to point B, keeping them safe and keeping them, you know, entertained and happy.

So, there’s a lot of moving parts. Definitely.

A lot of other mission critical things that have to be considered.

6:37 Infrastructure Challenges for AI Models

I’d like to explore the infrastructure a little bit more. We talked about how the vehicle has to do a lot of different things, mission-critical things. It’s a constrained environment. There’s a lot of limitations.

What sorts of architectural choices can OEMs make to accommodate the environment for AI models while also fulfilling the primary mission of the vehicle, which is transportation and mobility.

So one of the things as OEMs make the transition to SDVs, there’s often a changeover from more like the CAN lower speed architectures networks within the vehicle to more of an Ethernet-based. So traditionally, you wanna see like more of an Ethernet backbone. And then we see anything from the more distributed types of network with distributed ECUs to domain-oriented architecture to even a zonal architecture. In all of these cases, you generally have the problem of either bringing the model to the data or bringing the data to the model.

And so in those cases, you need to have the right solutions where you can actually deploy that model into that edge ECU node or even an MCU to run where the data is available. Or conversely, if you’re going to put the model into, for example, a gateway, you need to have sort of an agent to grab the data from the edge ECU and bring it to that gateway. So those are a few of the critical items. Generally, we also think that a service-oriented architecture is more amenable to deploying these kinds of models and providing the flexibility that’s needed to run these in the vehicle and to see that capability.

But also the need to run these in a containerized environment. Going back to that safety, mission critical type of viewpoint, we wanna make sure that any model that’s deployed is always going to keep the vehicle safe and it’s not going to consume too many resources that would otherwise prevent those mission-critical functions from being able to execute.

Yeah. So you really need a flexible platform where you can deploy models in general-purpose ECUs and not rely on specific high-end GPUs, etcetera.

Because those ECUs have to do other things as well, like manage the powertrain, manage the infotainment systems. Absolutely. So interesting. So there’s a lot of different challenges that have to be juggled.

So I can see how you need to have a much more sort of a tailored solution for bringing edge AI into vehicles.

Definitely. And that’s where, just to that point, the optimization becomes so important. Right? Resources are constrained and so you wanna make sure that the models are as optimized as possible without giving up the accuracy that you need for the model performance. Great.

So clearly, is an opportunity, but there are also challenges.

9:23 Use Cases Beyond ADAS

And as you’ve explained, SDV technologies can pave the way for bringing in more edge AI use cases. By the way, we haven’t really talked about use cases. I know people talk about ADAS and autonomous driving, and a lot of that is dependent on AI. What are some of the other use cases that is enabled by Edge AI? And then why really should OEMs care about it?

Yeah. Definitely. So a lot of the, we think, most interesting use cases are actually totally outside of ADAS because that’s a very well-trodden path. A lot of folks have done a lot of great work in that area.

But using some of the more traditional silicon, the traditional hardware that’s available in the vehicle, we think you can run more interesting models that will deal with personalization and certain types of features. So I can give a few use case examples. Certainly tire wear and tire management can come into play from a safety perspective. And so there are some traditional physics-based models as well as taking data from other sources.

So certain driver behaviors where you can even personalize these tire wear analysis models or the hydroplaning type of models as well, making sure you have sufficient grip on the road. Providing an early warning to both the user or to other systems in the vehicle to change the traction control or things like that. Another place is in personalization. So things like how a user might actually interact with an ADAS system. So this can enable new features over time, for example.

I may have certain hardware in the vehicle that is capable of supporting a new model, but I don’t have time or it didn’t exist. I didn’t know of it at the time that I’m actually releasing the vehicle into start of production. These platforms can bring the ability to deploy new models to the vehicle and provide even more personalization. So where there’s maybe a personalized distraction.

So depending on who the user is, I may want to change the frequency or the way a particular distraction tone plays depending on whether I’ve been distracted more immediately leading up to the current moment in time. There’s a few others as well. There’s certain regulations that are coming out.

Certain regulations in the EU around automatic leveling of headlights and different things like that are gonna require these kinds of capabilities. And you can add additional features on top of that while you may be able to meet the letter of a given regulation. Now you have the capability to add a new function that can dynamically adjust the headlight even while driving, for example.

We’re starting to see, you know,ChatGPT or other types of LLM-based models also come into the vehicles. Is there any room for those types of LLM based models to come in, you know, aside from, you know, asking ChatGPT where the nearest coffee, you know, place is and and navigating to it? Yeah.

Yeah, absolutely. And this is what’s driving a lot of the excitement in the space for sure. And so there’s a variety of different LLMs and use cases for those. There’s the traditional chat assistants, which I think a lot of people are playing with, to varying degrees of success and enthusiasm for sure.

These are often running into IVI. And so depending on the particular use case of an LLM, you may want to run it on a different ECU in the vehicle. One other use case for LLMs that we’ve seen that we think is particularly compelling is around the area of cybersecurity. A lot of times the intrusion detection models have very discreet, very numerous rules for potential intrusions and require a lot of data to be transferred back to the cloud as well as a lot of false positives.

With LLMs, we’re finding some model vendors that are able to significantly reduce those false positives as well as cover a lot wider range of threat pathways, basically, threat vectors, and can be simplified into these models. One concern, of course, is, you know, what constitutes an LLM and can it run in the vehicle? And this is where a lot of that optimization know-how comes in.

Of course, there are SLMs that are continuously coming out. SLMs being Small language models.

Yes, exactly. Exactly where you draw that cutoff, I think is an open question.

But at the end of the day, as compute continues to increase and especially neural computing GPU type of capabilities in the vehicle ECUs, as well as the model compression and bringing down or compressing the capability of those traditional foundational LLMs into smaller and smaller tools, I think there’s a convergence that we’re going to be seeing in the next couple of years where these tools in the vehicle will become a lot more capable.

13:54 Sonatus’ Role in In-Vehicle Edge AI

Excellent. Let’s talk about Sonatus and Sonatus’s role in this evolution. I know, we are, a leader in SDV technologies and solutions. How does that play into the things that you’ve talked about?

Yeah. So Sonatus has a number of different products that kind of fall into these different categories. For us, it really starts around our Foundation product, which is focused on the EE architecture in the vehicle. It’s basically dealing with controlling the network traffic and ideally dynamically changing aspects of the network in order to optimize for the particular use case or the needs of the of the vehicle configuration.

There’s also our Collector product which basically enables customers to collect data from the vehicle kind of when and as needed. So rather than having to get everything or get nothing, I can get exactly what I want when I need it and no other time. And those really are key to being able to deploy AI in the vehicle along with the next step, which is really when a customer wants to take action in the vehicle. So we have an Automator product which enables configurations to say when certain trigger conditions are true in the vehicle, then I want to do some kind of actuation or send some signal elsewhere in the vehicle for consumption.

And that’s one of those things that with the advent of AI models being deployed into the vehicle itself, now the spectrum of possible outputs or vehicle signals that you might use to communicate one place or another can expand quite a bit. So you can actually use the outputs of an AI model in the vehicle as if it was a generated signal in the vehicle and consume it elsewhere or send it back to the cloud for other purposes. Almost like a true agent. Right?

It’s connecting the virtual or the digital to the physical. Definitely. Absolutely. And then with, you know, AI in the cloud, of course, on top of that, you can run some more intelligence.

So things like those LLMs are a bit more human-like in how they behave and how they think. Now you start to add that capability to start to diagnose things that are happening in the vehicle and take more intelligent actions. Yeah. So that’s where we see the direction going.

Sure. And then of course, we have Updater, which is our own OTA. So that plays a role in, you know, updating models throughout the vehicle lifecycle.

Definitely. Not to be left out given short shrift, by all means, OTA solutions like Updater are really important for bringing that model to where it needs to run in the vehicle as well as to offer that containerized environment to protect all the other operations taking place.

16:22 Conclusion and Future Outlook

Well, that was a very informative discussion, Steve. I really enjoyed our conversation. Thanks again for stopping at the podcast.

Thank you, Sanjay. Appreciate it.

Well, there you have it.

We are looking forward to a future where vehicles are not just, platforms for mobility innovation, but also for AI-enabled innovation. Stay tuned for more episodes where we will dive much deeper into the intersection of SDV technologies and AI and how that’s truly going to transform the future of mobility.