Graphics processing units (GPUs) have emerged as the most dominant chip architecture for self-driving technology and now power many advanced driver-assistance system (ADAS)-enabled vehicles. With the advent of edge computing and machine learning (ML) processed directly at the source, new architectures, such as Google’s AI tensor processing units (TPUs), would dramatically expand autonomous cars’ computation capabilities, consuming a fraction of the power required by existing systems.
In 2004, Sandstorm, one of the first autonomous cars, developed by the Carnegie Mellon Red Team for the first DARPA Grand Challenge, carried over 50kg of computing equipment, including a four-processor Itanium server for path planning, a dual-Xeon computer for image processing from its LiDAR and radar units, and a Pentium III PC104 for drive-by-wire control.
Today, we carry more processing power in the pocket device that we take everywhere with us.
The Birth of the GPU
In 1999, Nvidia Corporation announced the invention of the GPU, defining it as “a single-chip processor with integrated transform, lighting, triangle setup/clipping, and rendering engines that is capable of processing a minimum of 10 million polygons per second.”
According to Nvidia, modern GPUs process more than 7 billion polygons per second. A single Nvidia Quadro RTX 5000, with 13,600 million transistors, can boast processing power up to 348.5 GFLOPS.
GPUs gained massive popularity in the past 10 years and are now in use in many applications, including gaming, computer-aided design, high-frequency trading, DNA and RNA sequencing, blockchain technology, and cryptocurrency mining.
As connected and autonomous vehicles (AVs) require massive processing of graphical data and maps, GPUs are the obvious choice for many of the tasks of operating an autonomous car.
In 2017, Nvidia announced a partnership with Toyota to use Nvidia’s Drive PX-series artificial intelligence platform for its AVs. The Drive PX Pegasus system, based upon two Xavier CPU/GPU devices and two post-Volta (Turing)-generation GPUs, provides a peak performance of 8 TFLOPS. Tesla uses a custom version of the same platform for the ADAS and autopilot features on its electric cars.
The upcoming model of Nvidia’s Drive series, the AGX Orin, is a 17 billion-transistor chip, almost double the transistor count of the current PX Pegasus and continuing the trend of large, very powerful automotive SoCs.
TPU Cores: The GPU Companions for Machine Learning
In 2016, Google announced the development of a new processor architecture for “deep learning inference” called the tensor processing unit.
According to the blog post, “TPU is tailored to machine-learning applications, allowing the chip to be more tolerant of reduced computational precision, which means it requires fewer transistors per operation. Because of this, we can squeeze more operations per second into the silicon, use more sophisticated and powerful machine-learning models, and apply these models more quickly, so users get more intelligent results more rapidly. A board with a TPU fits into a hard disk drive slot in our data center racks.”
From the beginning, Google’s TPUs were tasked with improving the accuracy of mapping applications such as Google Maps and Street View.
The second and third generation of TPUs were announced by Google in May 2017 and May 2018, respectively. The second-generation design increased bandwidth to 600 GB/s and performance to 45 TFLOPS, and the third generation doubled the previous-generation performance.
And in July 2018, Google announced the Edge TPU, a purpose-built ASIC chip designed to run ML models for edge computing.
The latest Nvidia graphics processors, including the Drive PX and Drive AGX series, also include several tensor cores for machine learning.
Cars Can Become the New Cloud and ML Processing Units
One of the advantages of the amount of computing power now common in connected vehicles is the possibility of using it for machine learning and massive data processing.
As discussed above, GPUs have many advantages over traditional CPUs in the fields of graphic manipulation and floating-point processing. During the operation of an AV, the GPU has to dedicate most of its computational resources to the task of driving the car, during idle times that computing power can be used for other applications.
Additionally, as connected and autonomous vehicles roam the streets and highways, they collect massive amounts of data about their environment, their systems, and the routes they drive on.
That enormous amount of data is a gold mine for machine-learning applications and services.
In a current machine-learning environment, data collected by edge devices is continuously sent to the cloud for processing, and new and updated algorithms are produced to train the processing units on those devices.
With the power of the processors in an AV, however, it is possible to use that computational power to run ML machines in the car itself. This way, each vehicle can train itself based on the data it collects.
And with the advent of the fifth generation of cellular networks (5G), a new use for the power of connected vehicles arrives. Those connected on 5G, with powerful features such as low latency and network slicing, could provide the means to create virtual cloud networks linking vehicles in proximity. Thousands of cars can become carriers, connectors, and on-the-fly storage for 5G infrastructure. Naturally, security is one of the main concerns. A network of highly connected vehicles needs to be protected against hackers and other threats, such as network failures and natural disasters.
