Deep Learning on a USB Stick
Back in April last year a company called Movidius launched what was the world’s first deep learning processor on a USB stick. Based around their Myriad 2 Vision Processor the Fathom Neural Compute Stick was the first of its kind, and—after initially offering 1,000 for free to researchers and smaller companies—a full rollout was planned for the tail end of last year. But then the company got bought by Intel and, apart from a brief update at CES, both they and the stick disappeared from view.
However that changed at the tail end of last week as Intel launched a rebranded version of the stick, the Movidius Neural Compute Stick.
The stick is based around the Movidius MA2150, the entry level chip in the Movidius Myriad 2 family of vision processing units (VPUs). This ultra-low power VPU enables you to add visual intelligence and machine learning capabilities in battery-powered products such as autonomous drones, or intelligent security cameras—at the edge, without a connection to the network, or the need for a cloud backend for your machine learning application.
“The Myriad 2 VPU housed inside the Movidius Neural Compute Stick provides powerful, yet efficient performance — more than 100 gigaflops of performance within a 1W power envelope — to run real-time deep neural networks directly from the device. This enables a wide range of AI applications to be deployed offline.” — Remi El-Ouazzane, VP and General Manager of Movidius
Machine learning development is done in two stages. An algorithm is initially trained on a large set of sample data on a fast powerful machine or cluster, then the trained network is deployed into an application that needs to interpret real data. This deployment stage, or “inference” stage, is where the Neural Compute Stick is useful.
Movidius have created a tool kit which will translate a trained neural network from a desktop environment — it can accept networks expressed using the Caffe deep learning framework — into an embedded application on the stick, which can then be optimized to run on the Myriad 2 VPU.
The ability to run these trained networks “at the edge” nearer the data—without the cloud support that seems necessary to almost every task these days, or even in some cases without even a network connection—could help reduce barriers to developing, tuning, and deploying machine learning applications. It could potentially help make “smart objects” actually smart, rather than just network connected clients for machine learning algorithms running in remote data centres. It could in fact, be the start of a sea change about how we think about machine learning and how the Internet of Things might be built. Because now there is—at least the potential—to allow us to put the smarts on the smart device, rather than in the cloud.
Unfortunately, at least right now, the SDK for the stick is x86 only. While there are some hints of expanding platform support in the coming weeks, its not clear whether that means operating system or processor support. Without ARM support the Neural Compute Stick won’t have as big an impact on the Internet of Things as it could, because for the most part the Internet of Things is built around the ARM processor rather than the x86. There seems little point advertising the sticks ultra-low power usage, while saddling it the high power requirements of the x86 platform. But, now Movidius in an Intel company, ARM support seems regrettably less likely.
The Movidius team will be showing off the Neural Compute Stick at the Computer Vision and Pattern Recognition conference this weekend in Honolulu, Hawaii. If you want to pick one up the stick is available for sale now, and costs $79. More information about how to get started with the stick is available on the Movidius developer site.
UPDATE: Movidius has released instructions along with a video showing how to set up and run out of the box object recognition examples with the Neural Compute Stick and a Raspberry Pi 3.
You can find further discussions around running applications using Neural Compute and the Raspberry Pi on the user forums. There are also instructions for installing Caffe on Debian Linux, which should work under Raspbian and replace the missing parts of the SDK.