An EE Times podcast by Sally Ward-Foxton
In this podcast, Sally speaks to Silicon Labs’ Matt Maupin and Dan Kozin about the company’s wireless IoT chips with AI acceleration, what use cases need this capability, how the accelerator fits into the device’s deep sleep modes and how AI works with the company’s software stack.
SALLY WARD-FOXTON: Welcome to AI with Sally, a podcast that takes a closer look at some of the most interesting technology stories on artificial intelligence and machine learning. We’ll hear about the latest in hardware and software that has a big impact on the world of AI. I’m your host, Sally Ward-Foxton.
SALLY WARD-FOXTON
Welcome to AI with Sally, a podcast that takes a closer look at some of the most interesting technology stories on artificial intelligence and machine learning. We’ll hear about the latest in hardware and software that has a big impact on the world of AI. I’m your host, Sally Ward-Foxton.
SALLY WARD-FOXTON
Welcome back everyone!
In this episode, we’ll be learning more about the wireless IoT chips from Silicon Labs with dedicated AI acceleration on-chip, based on an in-house matrix vector processor design. In this interview, we go into use cases for these parts but we also discuss everything from how the AI accelerator fits in to the device’s deep sleep modes, to integrating AI into Silicon Labs’ software stack.
You can hear that interview with SiLabs’ Matt Maupin and Dan Kozin, later in this episode.
But first… some news.
SALLY WARD-FOXTON
A spinout of LG Electronics is commercialising the Korean Electronics giant’s IP for AI acceleration in consumer electronics, robotics and automotive. LG spun out its semiconductor and IP divisions in 2020, and AiM Future still counts LG as one of its customers. The company’s NeuroMosaic IP is already shipping in LG robot vacuum cleaners and washing machines, and was also in a concept refrigerator demonstrated at CES. This fridge could change the colour of its doors when you knock on the glass, and blinks to greet you when you approach. AiM Future’s IP is scalable and is designed to handle multiple neural networks at once. The IP also supports learning at the edge, which the company said robot vacuum cleaners could use to learn about the furniture in your house during the 20 hours a day they’re not in operation. A new generation of the IP is expected next quarter.
If you would like to read more about AiM Future, I’ll put a link to my article on the podcast page at eetimes.com.
SALLY WARD-FOXTON
Ultra-low power microcontroller company Ambiq Micro has launched an AI SDK for its sub-threshold and near-threshold driven parts. Running at sub-threshold voltages can save tons of power, but it isn’t straightforward. Ambiq’s technology platform uses 50 to 100 design techniques to get around this, including analog, digital and memory design. Designs have to run slower at sub-threshold, but for Ambiq’s market in wearable devices, this isn’t such a problem. Recent innovations are increasing clock speed by introducing more operating points into the company’s dynamic voltage and frequency scaling schemes. The company has shipped high volumes of its microcontrollers into wearables already.
The Neural Spot SDK allows you to treat the microcontroller as if it were part of Python to get up and running straight away. And Ambiq’s libraries take care of power and memory configurations using presets, in case you are not familiar with sub-threshold hardware.
The company intends to submit benchmark results in the next round of MLPerf tiny. On the roadmap is also a dedicated MAC accelerator for AI inference and filter acceleration.
Read more about Ambiq Micro at eetimes.com.
SALLY WARD-FOXTON
Israeli chip startup Hailo released three new products this month. Their earlier product, the Hailo-8, was a 26 TOPS co-processor for industrial gateways and aggregators, but the new family is more of an SoC design for smart cameras, with CPU, DSP and Hailo AI accelerator. There are three parts in the new Hailo-15 series offering 7, 11 and 20 TOPS. Power depends on the workload, but some customers are running them below 2 Watts, according to the CEO of Hailo who I spoke to last week. These chips are for IP cameras, like security cameras or smart city type installations, and crucially, they are software compatible with the Hailo-8, and the software already supports transformers like ViT. The accelerator microarchitecture is apparently “similar but improved,” and we should find out more about that over the next few months.
To read more about the Hailo-15 series, head over to the podcast page at EETimes.com, where I’ve put a link to my story.
SALLY WARD-FOXTON
Wireless IoT chip company Silicon Labs, based in Austin, Texas, is famous for its system on chip designs which bring connectivity to millions of smart home and other IoT devices. The company is dipping its toe into AI acceleration, adding a matrix vector processor it calls MVP to a couple of high-end designs it recently launched. These devices have popped up in the MLPerf Tiny benchmark results a couple of times, so I wanted to learn a little more about it.
In a recent conversation with Matt Maupin, senior product marketing manager at Silicon Labs, and Dan Kozin, senior product manager at Silicon Labs, we discussed everything from use cases for wireless IoT chips with AI acceleration, to how the AI accelerator fits in to the device’s deep sleep modes, to integrating AI into Simplicity Studio, which is Silicon Labs’ integrated development environment for its devices.
The first speaker you’ll hear is Matt Maupin, the second is Dan Kozin.
SALLY WARD-FOXTON
OK, welcome to the show, Matt Maupin and Dan Kozin joining us from Silicon Labs today; we’re going to talk about the BG24 and the MG24. This is the only family of Silicon labs parts with dedicated AI acceleration today. Perhaps we could start by talking about kind of broadly what category of device is this that we’re talking about and what kind of applications will these go into?
MATT MAUPIN
Yeah, I could jump into that one. You know this device when you talk about the BG24 and the MG24; they’re two different devices that address different markets. So the BG24 is our Bluetooth and Bluetooth mesh device, and our MG24 is our multi-protocol device. So not only does it do Bluetooth- Bluetooth mesh, but it also does 802.15.4 technologies like Zigbee, Thread, and of course, Matter. This is really today our premier device, and what I mean by that is it’s the most feature-rich device, has the highest amount of flash and RAM.
And really, a lot of differentiating features, you know, obviously AI/ML is just one of them. It’s got things like a +20 dBm output power for RF to make sure you have very long-range reliable RF performance. It has features like a 20-bit ADC, which I’ll talk about a little bit later. And then there’s a lot of other ones. On there as well. You know it- it that is something of what we call secure vault, so it offers PSA 3 certification.
So you know, really our high-end device, and because of that, it addresses a lot of markets. So this addresses things like smart home, connected medical, smart cities, commercial, etc. So that’s one of the reasons that this device is an ideal device for this AI/ML accelerator is because it can address so many markets; it just opens up more to AI/ML at the edge.
SALLY WARD-FOXTON
Yeah, great. Why do we need a dedicated AI accelerator in a kind of wireless IoT chip like this? I mean, because I feel like we can do a lot of, you know, a reasonable amount of AI just with the Cortex M33 that’s already on chip. Why do these particular markets need dedicated AI acceleration?
DAN KOZIN
OK, hi, this is Dan Kozin, and I’m helping out Matt on this. So because we’re an SoC, right? So we have we do the wireless and the ML on the same chip; putting an accelerator in there helps us offload some of the neural network processing and really allows for the wireless to work without any delay, without any interruption, and the inferencing can occur at the same time.
SALLY WARD-FOXTON
OK, so why use a homegrown accelerator? Why not license something from ARM or somewhere else in the industry? And why did you decide to develop this in House?
DAN KOZIN
Well, so we’re… because we cover so many applications. This isn’t purely just for ML. We have other applications that our vector processor can do, and so we’ve been, you know, we took an approach of having a little more general capability, and yeah, and so we’re able to accelerate ML as well as other operations.
SALLY WARD-FOXTON
What kind of other operations might we be talking about in applications like these smart home things, for example?
DAN KOZIN
So, so that was the- the accelerator was also targeting location awareness and angle of arrival, angle of departure kind of applications. So being a little more generic in that way.
SALLY WARD-FOXTON
So it’s more like a general vector processor that can be used for both, depending on the application, I see…
DAN KOZIN
Right.
SALLY WARD-FOXTON
Can you tell us anything else about how the vector processor works? It seems like it’s fairly generic; it’s not terribly specialized, even towards AI. Maybe you could tell us a bit about how it works. How big is it?
DAN KOZIN
So in terms of it, it is a matrix vector processor. And it’s a co-processor, so you can offload the operations, let the MCU do its work, and offload those operations. So what we’ve done is we’ve implemented some of the standard, some of the Tensorflow kernels at that layer, and then are able to offload that whole operation and let the MCU do its work or go to sleep.
SALLY WARD-FOXTON
I think your published material says for the AI workload, and we’re talking about four X faster with one sixth of the power versus the just running the AI workload on the M33. I wondered whether I can get both of those numbers at the same time. Is it 4X faster and 1/6 of the power, or is it either/or?
DAN KOZIN
Yes, and that’s how… So what happens is the operation gets offloaded to the accelerator. The accelerator runs faster on its inference, but while it’s running, the MCU can go to sleep. So you’re getting that increased speed, and your you’re reducing the power because you’re able to go to sleep faster, so it is an and: you get 4X faster and 6X power.
SALLY WARD-FOXTON
Perfect.
DAN KOZIN
Based on applications of course.
SALLY WARD-FOXTON
Of course. Yeah. I mean, speaking of the sleep modes, the– this family is part of the EFR32 series and one of the selling points of that series is the various levels of deep sleep and the five different sleep modes. For those of us that are old enough to remember Energy Micro and all the different various deep, deep, deep, deep sleep modes. I wondered whether the AI accelerator or the accelerator block… which sleep, which sleep mode is it in? Or which sleep mode is it active in? Is it possible to have the M33 asleep while this is awake and vice versa, or are they both in the same category.
MATT MAUPIN
Yeah, I’ll chime in a little bit on that, and then Dan can add if he wants to. But I think one of the critical things here is I like to look as at the RF as a perfect example of that. You know, one of the things we do to get low power RF is a lot of the, in fact, in the system, if you’re familiar with our different EM sleep modes, is there’s a lot of things. Either the radio could do, the peripherals can do while the MCU is in a sleep mode or an idle mode, and the same things here. So you know, if I’m doing something specific, the MCU can be in that sleep mode as Dan mentioned before. And then, of course, when I need to transfer data or do something else with it, I can wake up the core. So you know, that’s really one of the things we’re doing to get this low lower power, and it’s very similar. and then, of course, we look at this as the whole system. A lot of times, you don’t think about, you know, maybe that’s making that much a difference. But you know, if you’re several milliamps with that core on, and I’m reducing that, I’m not drawing that power during that time frame. And of course, we also work on, you know, trying to make sure that we wake up very fast, so that’s not really specific to the AI/ML; it is specific to our system and our low power, so the capabilities of the MCU going to sleep very fast, waking up very fast, and then there’s a lot of things we can do transferring data between different peripherals while the MCU stays asleep. So that’s sort of been an architectural consideration that we do with everything to make sure that we have that lowest power, and this follows that same suit as well.
DAN KOZIN
So when the MVP is operating, the CPU will go into its EM1 mode, so we don’t support all the different modes. There’s; still, it’s still active because it will need to wake up quick enough and send the data back, you know, and then move the data into the processor to the continued inferencing.
MATT MAUPIN
Yeah. Well, if you’re not familiar, EM0 is really our active mode. So there is a savings from EM0 to EM1 to EM2, etcetera. So you are saving power even though you’re in EM1; you’re not in a true active mode with the MCU.
SALLY WARD-FOXTON
I see. Uh, that sounds cool. So it’s not that you’ve added extra modes; it kind of fits into the sleep modes that you already you already had.
MATT MAUPIN
Yes, exactly.
SALLY WARD-FOXTON
Yeah, perfect. OK, maybe we can talk a little bit about software as well. Software obviously is absolutely crucial once you decide to add dedicated acceleration, you need to build an SDK, or you need to build a software stack specifically for that. How are you doing on that front? And I wondered whether the SDK that you got for AI is part of simplicity studio or is it on its own.
DAN KOZIN
That’s a great question. So we engaged a network of partners with our, with our solution. We’re- we’re TensorFlow native, so in our SDK we support the TensorFlow micro-kernel, and so anybody have, you know, building a TensorFlow Lite model can run it in on the chip. The way we look at our software offering though is that we look at the skills of our developers and bucket it into sort of three main buckets. There’s the experts, that really know TensorFlow, they know Python, they’ve been familiar with it, and they can build their own models, and they understand all the ins and outs of doing it properly. On the whole other side of the spectrum, there we call them the solutions, where basically it’s people who want the ML capability but they really don’t have the staff, they don’t have the understanding, they really would like to have a black box solution.
And in the middle we’re calling, we call them explorers, people who sort of know the concepts but really need a curated experience. You know, sort of like the full workflow from gathering data all the way to creating the model library. So we offer options with partners that are integrated with our SDK across that whole spectrum of users, and if you look at our landing page it and we just keep on building those offers. So you can look at our landing page and then see how we segment by application, use your skill and then software offering.
Directly within Simplicity Studio, we support that expert right, the one that knows TensorFlow knows Python and we have a couple, we have two different sort of flavors. One is we have a open source self-serve, community supported toolkit, we call it the MLTK. It’s for the TensorFlow developer and it really is, you can call it sort of like a reference package of an end to end from training to deployment, set of Python packages. The MLTK, that’s something that we offer for the for the experts, but again, you know we partner with Edge Impulse, SensiML, a lot of the standard players, Sensory, MicroAI, AIZip, AI Tad, Neuton. All those are integrated with our SDK and are off and are options people can use.
SALLY WARD-FOXTON
If I felt like I am an expert and I just wanted to use your SDK, is it possible to just use your SDK on its own without going to Edge Impulse and so on? It’s a full set, right?
DAN KOZIN
Yes, full set, full offering so TensorFlow native, come with your TF Lite file and run the profiler to see that it fits, and yeah and load it up and give it a try.
SALLY WARD-FOXTON
Uh, do you have a large model zoo that I can take a look at? Like how many types of what different different types of models do you have up and running?
DAN KOZIN
We don’t, we don’t offer a specific model zoo. We kind of offer it, but we offer it as…. You know, it’s sort of like a tutorial training through the MLTK of how to build, say, an audio signature application.
MATT MAUPIN
Yeah, I’ll chime in a little bit on this. I’m because I’m not, I’m more of a hands-on person as far as trying to understand things. And actually yesterday we did a tech talk on AI, ML and what I loved about it is one of our FAEs actually demonstrated SensiML and how they could use that to actually get data, create a data set, train device and test the device. And it was really interesting seeing that and we did that with one of our eval boards with a fan hooked up. So we was using that an acceleration sensor to take the data, log that data and do the examples, and then that information was able to be exported into studio, to actually import into the project. So for me you know this that was made it more real, and also the fact that this stuff is pretty new, so there’s going to be a lot of customers that may not really understand this and that’s gonna help them get started. So as Dan commented on, we really address the whole, the experts all the way to the people that want to just take a deployed system. Maybe they just want to do voice tagging or something like that, and they can just sort of take a complete package. So it’s really great to start taking this and advancing it at the edge.
SALLY WARD-FOXTON
Super cool. And do you have any feeling yet for what kinds of models people are running, are we looking at person detection, sound detection, keyword spotting like audio or cameras or you know any particular areas that been popular so far?
DAN KOZIN
So the areas that we sort of fit well into and we have a lot of sort of demand, starts at that sort of low sensor rate, right, the predictive maintenance IMU sort of type of input going up to audio signature. I call it audio signature, it’s non voice audio signature. So it’s like gunshot or glass break or you know, even squeaks. Right. And then and then moving up into voice where it would be a wake word or a or a voice command, and that’s not like sentence recognition. It’s just, you know, words, you know, small vocabulary words.
SALLY WARD-FOXTON
OK Google or whatever.
DAN KOZIN
Yeah, right. Something like that. And then and then we’re we’re just sort of, with the power of our processor, the xG24s, we’re just able to scrape into the sort of image segment with low res kind of IR camera input. So maybe 96 by 96, you know grayscale or, you know, low frame rate, for things like person detection or occupancy detection, people flow something like that. So that that sort of covers the range of the processor that we have in the applications that we conserve.
SALLY WARD-FOXTON
Can you tell me anything about what is on your road map for AI? Are you planning to add dedicated accelerators to more parts in the range, and if so, which ones? Or are you working on a new generation of accelerator? Where are you going to go from here?
MATT MAUPIN
Yeah, I could talk a little bit about that. You know, obviously you mentioned the xG24, we have it today is shipping and in production. We have announced the SIWX917 – this is a Wi-Fi device and we do have information. So if you look on the web page you will see that it also does have an AI/ML accelerator. So those are really, the only two devices that we’ve announced today and you know, I think the key here is that we really believe that this is something that’s going to be important in the future and I think a great example is what we did with our secure vault in 2019, we first brought that out and it was leading edge. And we sort of did that honestly before a lot of customers were asking for it. You know, obviously we had some, but it wasn’t really what I would consider mass market appeal. And since then, you know, now we’re in 2023, that was four years ago. This has become more critical and we believe this is going to follow the same path. So this is something that we believe is critical. Obviously it’s going to be on certain devices. If I’m doing if I have a device that is very niche, for example, if I do something that’s very specific to maybe a shelf tag, maybe it doesn’t belong there, but as you go up the scale, it definitely is something that we see very important and we’ll be continuing on.
The other thing I wanted to sort of chime in on a little bit is you know Dan was talking about some of the applications and you know, obviously we were engaged with a lot of customers and that are doing those typical applications. However, one of my favorite is something that I hadn’t really thought about or seen before, and I can’t really go into too much detail, so I’ll keep it at a high level but…we have an existing customer that basically is using this to enable a new feature on a product that is already wireless today, but it’s really helping them enable something that can add some differentiation. And what I love about it is they also took some other features, for example, one of the things I talked about was the the 20 bit ADC, so it’s a very high resolution ADC.
MATT MAUPIN
So this device was… think of it as having a traditional method of detecting a failure, as a safety feature, and now what they’re doing is they’re, with ML at the edge, they need a very fast response, less than 25 milliseconds. They’re able to add this new feature where they could actually detect this earlier and not only that, but now they could start to use this data to actually get more information and improve their product, and it may be may be able to do other things because of the data set they’re getting now. And again, what I loved about it is, you know, the fact that we offered a couple of these different features – they could have implemented this today or you know on their current product, but it would have taken probably 3 chips. You would have an analog front end, you would have had an MCU, higher end MCU with some type of AI offload or ML offload, and then the wireless device. Now we’ve integrated that all into a single device. So they’re able to hit a price point that they couldn’t do before and really add differentiation. And it’s not one of those typical applications that Dan talked about that we see a lot, this was something that to me was really surprising. I didn't think about it and it’s really interesting to see how our customers are actually going to come up with different features that they couldn’t do before because of this integration.
SALLY WARD-FOXTON
Yeah, I think we’re only starting to scratch the surface of the types of applications that ML at this scale can go into – I hear about new ones every day and it’s very exciting. I’m sure you do too.
DAN KOZIN
Yeah, it’s such a horizontal play. And there’s a, I’m just going to throw another sort of innovative use of ML again, and it’s in our MLTK people can look at it, but typically people think of ML as just pattern matching, like, look for some patterns and then do some actionable event, but this is sort of turning it around or it’s saying where it’s using ML to create a unique signature of a voice print or fingerprint. And so it’s sort of taking that that that neural network and you know how it will determine… Typically it will determine like, oh the finger, you know it, this is an apple that’s sort of 70% looks like an apple. Right. It’s not exactly an apple, but it looks enough like an apple. Well, that creates a signature, turning that around and being able to create a unique signature based, a unique imprint, unique input is just another innovative way to use the you know, neural networks. And so we have some in fact SensiML is doing a demo where they’re using voice authentication, prototyping in their smart door lock with that functionality.
SALLY WARD-FOXTON
That’s super. That’s super cool. I wonder how much horsepower it takes to do voice authentication, though. It sounds like a complex one.
DAN KOZIN
It’s just using the same neural network. It’s using the same inferencing. It’s exact same power as detecting, sort of like some audio signature.
SALLY WARD-FOXTON
That’s super cool.
SALLY WARD-FOXTON
Thanks so much Matt Maupin and Dan Kozin from Silicon Labs, for the insight into the MVP and what Silicon Labs customers are doing with it.
That brings us to the end of the episode. Please tune in again next time to hear more news and views on AI, machine learning and the technologies that enable them.
If you are listening to this on the podcast page at EETimes.com, links to articles on topics we’ve discussed are shown on your left.
AI with Sally is brought to you by AspenCore Media. The host is Sally Ward-Foxton and the producer is James Ede.
Thank you for listening!
