Moving data around uses energy, but so does storing and processing it. And as more data is moving back to the edge for a variety of use cases in many different environments, there’s a need to store and process more of it without consuming more power.
While edge computing and internet-of-things devices aren’t as power-hungry as the massive data centers used for cloud and high-performance computing environments, more smarts at the edge mean more processing and more local data storage with their own minimum energy requirements. Mix in 5G or Wi-Fi connectivity, and power consumption becomes an even more critical element of the design process.
Power sources at the edge must also be reliable, given that many devices are deployed in remote areas and harsh environments to operate without regular human attention for long periods of time. Reliability is also important to meet functional safety requirements and allow for artificial intelligence to do accurate and fast inference to support critical decisions in real time.
Edge device components are getting smarter
AI is driving a great deal of edge computing deployed at the edge, further enabled by the rollout of 5G networking.
More specifically, the application of computer-vision technologies is fueling the proliferation of a wide variety of devices for many different industries. They can gather more information and derive insight to make smarter decisions to improve the bottom line of a business or the quality of life for citizens.
Regardless of the application, computer-vision technologies that capture digital images, video, and other visual input all have core elements that consume power. There’s always a camera or other type of sensor that acquires input, a processor, memory and storage for captured data, and networking capabilities to send and receive data. Everything works together to enable pattern-recognition algorithms so the computer vision can mimic the human brain and understand what it sees. AI in the form of machine-learning and deep-learning methods allow for a device to recognize a wide variety of visual information and act on it.
Some applications require very little processing power, memory, storage, and power because the tasks are quite simple in nature, while others are more complex in nature. Some edge devices do more work locally, while others send back the bulk of the data to a central point from processing. It all depends on the use case, and there are many.
More varied AI workloads put pressure on power consumption
Some edge computer applications are the evolution of uses that have existed for decades, while others were only recently possible thanks to advances in AI, memory, and networking technologies.
Security and surveillance in workplaces and public venues aren’t new, but computer vision allows for more intelligence and better detection capabilities. This is useful in typical security scenarios because an edge device can scan large crowds to detect dangerous or suspicious behavior to prevent crime or injury or protect a restricted building from unauthorized entry. Being able to detect people in terms of numbers and location can also allow for smarter environmental controls, so air conditioning can be turned on in a crowded room or heat kept low in a space that isn’t being used.
In a retail environment, computer vision in an edge device can aid in theft prevention, as well as enable customer behavior observation that provides insight into buying patterns and anticipates consumer activity, including dissatisfaction that leads to complaints and churn. In a retailer’s warehouse, a trained AI system can address supply chain inefficiencies because machine learning enables it to infer what’s in stock and how much. On the factory floor, computer-vision technology plays a key role in Industry 4.0 to draw real-time information and insights, reduce product defects, and improve safety.
Not all edge devices are in controlled environments, however. Some are literally out in a field for agricultural uses to improve crop-yield analysis or monitor livestock. Others are part of city infrastructure to support smart mobility to improve the routing of public transit and support autonomous vehicles. This is where the tasks for edge devices become complex, which means more data processing and storage and potentially more power. Even if a vehicle isn’t fully autonomous, it is full of endpoints with cameras and sensors to collect information from other drives on the road so it can learn from their behavior and allow for an autonomous vehicle to fully see its surroundings so that it can spot traffic signals and change lanes safely.
When decisions need to be in real time, the speed and reliability of 5G networking is essential, as is sufficient power to all the electronic components that must rapidly respond to input, process the relevant data, and act accordingly.
Emerging and incumbent memories are both vying for the edge
AI and 5G are intertwined for many edge-computing use cases because they must work together to enable the responsiveness required for many applications. But these devices are also becoming more memory- and storage-hungry. Just as 128K turned out to be insufficient for most computer users, even simple devices at the edge are doing more processing locally rather than sending everything back to a central cloud.
There are many options for edge and IoT devices for memory and storage, and there’s been a lot of chatter about how emerging memories can meet their requirements. Non-volatile resistive random-access memory (RAM) shows promise, as it scales easily and demonstrates greater energy efficiency than DRAM. Its multi-bit storage also improves the accuracy of neural-network inference, which is a vital component of AI. Magnetoresistive RAM also has potential at the edge because it can run at lower voltages in applications in which high-performance accuracy isn’t necessary but energy efficiency and memory endurance are critical.
But there are many indications that tried-and-true conventional memories will be the most suitable for edge devices, even for AI tasks. NOR flash is known for its reliability and longevity, which is why it’s a key memory for automotive applications. Today’s intelligent and autonomous vehicles can easily be seen as one large moving edge device made up of a series of small edge devices in the form of sensors and cameras to support advanced-driver-assistance-system functions as well as on-board entertainment. Some of these functions also require “instant on” capabilities, as do many other edge devices that might sleep for periods of time and switch on only when needed. Overall power consumption in a vehicle is also a consideration.
A lack of moving parts in a storage device reduces energy consumption, which is why NAND flash is appealing from a power and capacity point of view. Whether it’s the eMMC or UFS for factor, flash has been qualified for automotive applications, and its established reliability makes it an obvious choice for other edge applications. However, it’s not the fastest option, and some AI tasks have real-time decision-making requirements, especially those related to autonomous-vehicle traffic management.
On-chip SRAM may be enough for edge devices if it’s just for inference; embedded SRAM has shown potential in system-on-chip designs for wearable and IoT applications where power consumption, heat dissipation, and battery life are all considerations. SRAM may also address important security functions at the edge for IoT devices in the form of an SRAM physical unclonable function as the trust anchor for any device.
SRAM is appealing where speed is important, but nothing beats the speed of DRAM. Its performance and capacity are increasingly in demand as edge applications become more complex, although it does come at a cost. However, low-power DDR DRAM (LPDDR), which is generally used in smartphones as well as automotive applications, may become more ubiquitous at the edge. The latest iteration of LPDDR5 has an Adaptive Refresh Management feature, which helps the memory device adjust in more stressful operating environments, including more extreme temperatures — essentially, it’s a mechanism that defines a reliability level so the host or software can know what to expect from the device.
But as the name implies, LPDDR5 is about getting more performance while consuming less power, and although smartphones tend to be the first market for this type of memory, the combination of improved performance, power, and flexibility could make DRAM more ubiquitous at the edge, even if it does tend to be one of the more expensive memory options.
Ultimately, the required inference and computational tasks combined with the type of device will determine what memory is most suitable, but balancing power constraints and performance will always be the deciding factor at the edge.
