AI algorithms consume a lot of computing resources, and it used to be regarded as impossible to perform AI computing in embedded systems using microcontrollers (MCU). Nowadays, with the progress of technology, it is no longer the case. This article will show you the development status and solutions of AI applications on MCU to understand the latest technological development trends.
It is the general trend to use MCU to perform AI computing
When it comes to AI, its related technologies also include machine learning and depth learning. These technologies require a large amount of computing resources and are usually related to powerful processing technologies such as CPU, GPU and FPGA. However, with the development of AI technology towards cloud to the edge computing, AI computing engines will allow MCU to push embedded applications to a wider world. At the very least, it will enable embedded design to improve the real-time responsiveness capability to cyberattacks and improve the security of devices.
At present, many electronic products on the market are designed by MCU, which has been widely integrated into people's lives. Nowadays, with the rapid development of AI, more and more smart products have AI functions, such as Internet of Things (IoT) devices, smart household appliances, smart speakers, etc., which prompts more companies begin to attach importance to the application of AI characteristics on embedded devices using MCU, and pay more attention to the development of MCU combined with AI.
The application of AI on MCU has considerable advantages, i.e., reducing power consumption and cost of products, accelerating the release of products, and directly processing at the front end without cloud computing, thus improving the efficiency of AI computing. However, due to the weak computing power and low computing frequency of MCU, there is an obvious gap compared with the computing power of CPU.
In addition, MCU lacks modeling and training tools, so to realize AI computing on MCU, PC-side tools are usually required, and the model after establishment is usually large and unsuitable for MCU use, and there are still some difficulties in the application of integrated process and development tools.
New technology solves the problem of running MCU in AI computing
Facing strong market demand, many manufacturers have put their efforts into this, solving the problem of performing AI computing on MCU. At present, many solutions have emerged, as in applications such as object recognition, voice-enabled service and natural language processing, which have enabled MCU to have AI algorithm capability and can also provide higher accuracy and data privacy for battery-powered devices in IoTs, wearable devices and medical applications.
Among them, the first most common method to let MCU perform AI algorithm involves model transformation for a series of Neural Network (NN) frameworks (such as Caffe 2, TensorFlow Lite and Arm NN), and models and inference engines for cloud training on MCU. At present, there are some software tools that can obtain pre-trained neural networks from cloud and optimize MCU by converting them into C code.
Optimization codes running on MCU can perform AI functions in voice, vision and anomaly detection applications. Engineers can download these tools into MCU configuration and run inferences of optimization neural networks. These AI tool sets also provide code examples based on neural network AI applications.
The second method bypasses the need for pre-training neural network models borrowed from the cloud. Designers can integrate AI libraries into MCU and incorporate local AI training and analysis functions into their codes. Subsequently, developers can create data models based on signals obtained from edge sensors, microphones and other embedded devices, and run applications such as predictive maintenance and pattern recognition.
The third method is to provide AI-dedicated coprocessors to enable MCU suppliers to speed up the deployment of machine learning functions. Coprocessors such as Arm Cortex-M33 can use popular APIs such as CMSIS-DSP to simplify code portability, thus allowing MCU and coprocessors to be tightly coupled to speed up AI functions (such as co-relation and matrix operations).
AI function package for computer vision
In order to help STM32 MCU support AI function, STMicroelectronics has introduced the FP-AI-VISION1 function pack for the high-performance STM32Cube, which can be used in AI application of computer vision. FP-AI-VISION1 has a computer vision application example based on convolution neural network (CNN). FP-AI-VISION1 is composed of software components generated by X-CUBE-AI expansion package, and complemented with application software components dedicated to AI-based computer vision applications.
Examples of applications provided in the function pack include food recognition (which can identify 18 classes of common foods) and person presence detection (which can identify whether there is a person in the image). FP-AI-VISION1 uses STM32_AI_Runtime neural network libraries to realize advanced computer vision applications. The libraries are based on pre-trained models and generated with the X-CUBE-AI expansion package of STM32CubeMX tool.
Provide complete functions and examples to speed up product development
The FP-AI-VISION1 function pack demonstrates the integration of two types of neural network models: 32-bit floating-point model and 8-bit quantized model. It also demonstrates model integration in different memory configurations (relying only on MCU internal memory or using external memory additionally).
The FP-AI-VISION1 function pack has an image preprocessing library STM32_Image, which can be combined with specific hardware boards to develop specific computer vision applications. It includes various processing functions to process the contents of frame buffers. The FP-AI-VISION1 function pack also includes a driver for USB webcam, which can be used to create image and video data set collection, capture images into frame buffers, preprocess the contents of frame buffers, and running a framework for neural network inference, which can perform real-time tests on the host.
FP-AI-VISION1 can also be connected to the B-CAMS-OMV camera module bundle (recommended) or the STM32H747I-DISCO Discovery board of the STM32F4DIS-CAM camera daughter board (legacy only), and can act as a USB video camera (UVC) device when connected to a host.
With complete firmware, FP-AI-VISION1 can develop computer vision applications on STM32 microcontroller. It has integrated examples based on floating-point model and quantized models, and libraries that can test, debug and validation embedded applications, and has free and user-friendly license terms, convenient for manufacturers to develop products.
Conclusion
The application of AI has become the hottest trend of technological development. The combination of AI and Internet of Things application (AIoT) will turn over the application modes of various products, truly realize the more intelligent interconnection of all things, and completely change the way of human life. At present, it has become a reality to perform AI operations on MCU, which opens a broader door for more embedded systems to apply AI, and the future development of technology will be more expected.