With artificial intelligence (AI) and machine learning (ML) making their way from the realm of science fiction to reality, we now need an easy way to prototype this type of system. A desktop computer may suffice for AI/ML ― or even a single-board computer like the Raspberry Pi ― but what if all you want is a simple plug-in device to make your system much faster and more capable?
As it just so happens, you have multiple options from which to choose, including Google's Coral TPU Edge Accelerator (CTA) and Intel's Neural Compute Stick 2 (NCS2). Both devices plug into a host computing device via USB. The NCS2 uses a Vision Processing Unit (VPU), while the Coral Edge Accelerator uses a Tensor Processing Unit (TPU), both of which are specialized processors for machine learning. See below for stats on both of these devices.
Note that this article refers to the Neural Compute stick 2, which acts as a USB accelerator, not the similar-looking — but functionally much different — Intel Compute stick.
Coral Edge Accelerator
- ML accelerator: Edge TPU ASIC (application-specific integrated circuit) designed by Google. Provides high-performance ML inferencing for TensorFlow Lite Models.
- USB 3.1 (gen 1) port and cable (SuperSpeed, 5Gb/s transfer speed)
- Dimensions: 30 x 65 x 8mm
- Price: $74.99 (Arrow)
Intel Neural Compute Stick 2
- Processor: Intel Movidius Myriad X Vision Processing Unit (VPU)
- USB 3.0 Type-A
- Dimensions: 72.5 x 27 x 14mm
- Price: $87.99 (Arrow)
Comparing Processor and Accelerator Performance
Unlike computer CPUs of old, which you could compare numerically by clock speeds, comparing each processor/accelerator is a bit more nuanced and depends on how you intend to use them. You can find benchmarks, here for the CTA and here for the NCS2, albeit in different formats. Note that the output formats are different — time per inference vs. frames per second — but we can draw a few overall patterns for both devices.
First, the CTA works very well when added to a desktop CPU, speeding up performance by roughly tenfold, a figure that might be better or worse depending on the model you choose. The NCS2, per the above-linked testing, speeds things up by nearly seven times when compared to an older Atom processor. But the NCS2 had marginal results when paired with more powerful processors. Note that Google officially produced these CTA results, while the NCS2 results were independently produced and less detailed.
The NCS2 can theoretically perform at a 4 TOPS (trillion operations per second speed). Oddly enough, the CTA boasts the exact same rate. While both use different processes to produce ML results, one would have to assume that numbers in a more controlled head-to-head test would be comparable. The Intel claims up to an eight-times performance boost over the original Neural Compute Stick, so you'll want to go with the NCS2 if you have a choice.
Power-wise, the NCS2 boasts a low power consumption, though I wasn't able to find hard numbers to back that up. The Coral Accelerator, for its part, does list .5 watts needed for each TOPS. Users can set the CTA to a default speed or a maximum setting (2x default) if needed, though it's not immediately clear whether the .5W/TOPS number applies in both cases.
Notably, Google's documentation does warn of possible skin burns because of power transfer when the device is running at maximum speed, along with a lowered maximum ambient temperature. I say go ahead and run it in normal mode unless you really need the extra processing power.
Software support
The NCS2 can work with Ubuntu, CentOS, Windows 10, and other operating systems. It can support TensorFlow, Caffe, ApacheMXNet, Open Neural Network Exchange, PyTorch, and PaddlePadle via an Open Neural Network Exchange conversion.
The CTA does not come with Windows support, but it can run under Debian 6.0 or higher (or any derivative, such as Ubuntu 10.0+). Coral TPU can officially only run TensorFlow Lite models.
Size, design, and other considerations
With software, computing, and power consumption covered, what about the actual build? Frankly, both devices look incredibly cool. The CTA boasts a partially transparent body as well as what appear to be slots for heat dissipation. However, the NCS2's slick blue design, with its anodized blue body and integrated heat sinks, wins the style competition. Of course, appearance is a minor point.
What is important is that the NCS2, like the CTA, does get quite hot, and the clever heat sink design means you can grip it on the cooler integrated fins instead of burning your fingers on the middle if you need to handle it.
The NCS2 is designed to allow several to be used together for expanded processing power. You can arrange them neatly in a vertical USB hub. A single host computer can also run several CTAs, though you may have to find another way to hold each one. On that note, while each have a similar footprint, the NCS2 is close to twice as wide (14mm) as the CTA. Combined with the fact that it plugs in via a USB plug like a very large thumb drive — not through a flexible cable like the CTA — means that you'll have a hard time fitting the NCS2 into a lot of spaces. You can always opt for extension cables and hubs, but it's something to consider.
In the end, both the NCS2 and the CTA appear to be solid devices for edge computing applications. The NCS2 has a distinct advantage if you need to run on a Windows system or if you need to operate outside of the Tensorflow Lite framework. For its part, the Coral Accelerator has a more direct dev board and SoM analog in the form of the Coral Edge TPU dev board and module, possibly making it more attractive if you want to take your design to market quickly. With all the functional overlap between the two, either device would work well if you want to add AI/ML to your next Raspberry Pi or similar project.
