More computing is being done at the edge, and this is driving increased demand for on-board memory, storage, and processing capabilities to support more advanced artificial intelligence and machine learning done locally at the device level.
More computing is being done at the edge, and this is driving increased demand for on-board memory, storage, and processing capabilities to support more advanced artificial intelligence and machine learning done locally at the device level.
While early edge AI use cases included simple tasks such as sensing and detection for security and access purposes, internet-of-things devices are doing more inference locally — they’re no longer just collecting data to send back to a central cloud that does the heavy AI/ML lifting. Instead, these devices are doing local ML that can be shared and redistributed to other devices.
More complex AI and ML means more computing power is needed, and that includes increased memory performance and data storage capabilities.
Edge devices are making more complex decisions
Until recently, a lot of edge AI boiled down to sensors detecting something was wrong — someone was in the building who shouldn’t be, or a machine on the production line was overheating and about to fail. These systems were smart enough to understand something was wrong and alert someone to do something.
Today’s AI at the edge is getting far more intelligent as hardware enables IoT devices to do more. This in turn increases expectations of what can be done at the edge and thereby drives demand on hardware. Thanks to advances in processors, memory, and storage media, AI at the edge can do more complex inference/ML tasks, increasingly without the need to send data to a central cloud and back.
Beyond the evolution of emerging and incumbent memories, including flash technologies for storage, there are several other advances in technology that are enabling AI/ML tasks at the edge. Highly parallel GPUs have been adapted to execute neural networks, which allow for generalized machine learning that can be deployed at the edge, while the proliferation of IoT devices, including sensors, cameras, and even robots to collect data, are constantly revealing new opportunities for AI/ML capabilities that support a wide range of business objectives. Overall, neural networks and the AI infrastructure to support them have matured, including 5G connectivity that eliminates the latency to support local communication between devices and back to the cloud.
The high availability of today’s edge computing devices coupled with offline capabilities and decentralization means constant internet access is no longer needed to process data and perform AI/ML operations. Edge AI applications are now more powerful and can provide real-time insights because data can be analyzed locally and process diverse inputs such as text, voice, and video and make inferences independently. Not only are they trained to answer a specific question, but they can also answer new questions if it’s a certain type. This reduces the need to communicate with a central cloud but also enables a group of edge devices to conduct local ML operations and share those lessons back to be more widely distributed. In addition, the more they train a model at the edge, the more accurate the AI model becomes.
Maturing AI at the edge allows for more complex applications across a wide range of industries. Predictive maintenance in manufacturing environments was early application of industrial IoT and is becoming even more robust thanks to local AI/ML capabilities because sensors can detect problems even earlier and predict failure more accurately, not just due to wear and tear but also due to misconfiguration or misuse.
Facility access and management can be better controlled through AI, and not just for security purposes. Smart buildings can adjust temperature and lighting based on how many people are in the building, as well as allow more remote management of HVAC systems. Other energy applications for AI include using edge models to make generation, distribution, and management more efficient based on weather patterns, infrastructure and grid health, and historical data.
Other sectors benefiting from more advanced edge AI capabilities include health care, retail, and agriculture. But no matter what the use case or the location, more advanced AI/ML is putting pressure on the computing hardware itself, especially memory, so more can be done locally.
AI ubiquity favors proven memory technologies
Many different memories are being considered and used in edge devices that perform AI/ML operations — this a reflection of the diversity of edge AI applications.
In recent years, there’s been a great deal of discussions around emerging memories such as resistive-RAM (ReRAM) and magneto-resistive RAM (MRAM) for IoT edge devices that have created a whole host of new requirements for memory, but it’s increasingly apparent that enduring, legacy memories have just as much of a role to play to enable AI devices at the edge.
The appeal of MRAM for AI at the edge lies in its power consumption because it runs at lower voltages in applications in which high accuracy isn’t necessary, but memory endurance and energy efficiency are critical. MRAM’s nonvolatility also means it can store data without power, making it a suitable replacement for SRAM and embedded flash simultaneously, thereby acting as a unified memory. ReRAM’s appeal for AI at the edge is derived from its potential to mimic how the human brain learns and processes information at the neuron and synaptic level; ReRAM devices are significantly smaller and more energy-efficient than DRAM and HBM, which are employed in data centers for AI/ML operations.
But even though MRAM and ReRAM have characteristics that make them suitable for many AI edge applications, the increasing ubiquity of AI may mean that tried and true memory technologies will triumph as even basic inference operations become more complex, with low-power DRAM (LPDDR) memories making the most sense. Even older iterations such as LPDDR3 can support 4K, full HD, or 3D sensor video image processing in real time to support AI applications such as face recognition in security cameras or gesture control in public kiosks, as well as perform natural-language processing. Similarly, NOR flash will continue to be selected for some edge AI use cases due to its reliability and longevity while operating in harsh environments, both for processing tasks and data storage.
Other options for edge AI data storage include universal flash storage (UFS), including removable cards, thanks to the specification’s high reliability and favorable power consumption and stability characteristics. A UFS card also allows a host to send commands continuously, even while it’s transferring data for processing previous commands, which means an AI edge application can perform I/O operations with other applications simultaneously without impact on performance.
Overall, there are many opportunities for commodity memories to meet the needs of edge AI. After all, yesterday’s supercomputer memories are now in smartphones, which are also a flavor of edge devices. Earlier AI models that needed high-end hardware can now be handled using more mainstream memories in an IoT device thanks to miniaturization and semiconductor companies continuing to drive the costs out of the hardware.
Although some AI/ML operations will always need centralized, high-performance computing, the inference and computational tasks being done at the edge as well as device type will determine which memories are most suitable, with thermal characteristics and power constraints having a strong influence. Ultimately, there’s going to be a finite number of memories needed to support AI at the edge, be it emerging or the familiar incumbents.
