Benchmarking the Xnor AI2GO Platform on the Raspberry Pi

Headline results from benchmarking

Using AI2GO on the Raspberry Pi we see a considerable speed increase over the fastest times times seen with TensorFlow Lite and MobileNet v1 SSD 0.75 depth model in our previous benchmarks.

Part I — Benchmarking

A more detailed analysis of the results

Our original benchmarks were done using both TensorFlow and TensorFlow Lite on a Raspberry Pi 3, Model B+ without any accelerator hardware. Inferencing was carried out with the MobileNet v2 SSD and MobileNet v1 0.75 depth SSD models, both models trained on the Common Objects in Context (COCO) dataset, converted to TensorFlow Lite.

Inferencing speeds in milli-seconds for MobileNet SSD V1 (blue) and MobileNet SSD V2 (green) across all platforms, with the new Xnor binary convolution network performance shown for comparison (red). Lower numbers are good!

Some caveats around the results

In our original benchmark, and then again when we went back to look at TensorFlow Lite on the Raspberry Pi, we tried as much as possible to keep our models the same. Comparing apples with apples, rather than apples with bananas. We can’t do that here.

Detecting fruit on the workbench in our test image (left) 🍌🍎, using TensorFlow (middle), and Xnor (right).

Summary

The addition on these new benchmarks for the Xnor network hasn’t changed the overall result. The Coral Dev Board and USB Accelerator still have a clear lead, with MobileNet models running between 3× to 4× times faster than the direct competitors.

Part II — Methodology

Getting the AI2GO SDK and Model Bundle

You should go ahead and grab the AI2GO SDK from the Xnor site. You’ll need to accept the terms of service before the Xnor site will allow you to create an account and download their SDK.

Downloading the Xnor SDK
Pick your hardware platform
Select an ‘industry’ rather than a specific machine learning model
Selecting a ‘Kitchen Object Detector’ model.
The pre-tuned model.
Other model options.
Download the model bundle.
$ ls
LICENSE.txt libxnornet.so
README.txt xnornet-1.0-cp35-abi3-linux_armv7l.whl

Installing AI2GO on the Raspberry Pi

Go ahead and download the latest release of Raspbian Lite and set up your Raspberry Pi. Unless you’re using wired networking, or have a display and keyboard attached to the Raspberry Pi, at a minimum you’ll need to put the Raspberry Pi on to your wireless network, and enable SSH.

Everything you need to get started setting up the Raspberry Pi
% ssh pi@raspberrypi.local
$ cd xnor-sdk
$ pip3 install Pillow
$ pip3 install psutil
$ cd ~/kitchen-object-detector-medium-300
$ pip3 install xnornet-1.0-cp35-abi3-linux_armv7l.whl
Processing ./xnornet-1.0-cp35-abi3-linux_armv7l.whl
Installing collected packages: xnornet
Successfully installed xnornet-1.0
$

The benchmarking code

Our benchmarking code is a mash up between our original TensorFlow benchmarks and the sample code provided by Xnor, and is straightforward.

In closing

As I really tried to make clear in my previous articles putting these platforms on an even footing and directly comparing them is actually not a trivial task. While initial results around the Xnor models look extremely promising, the fact that the models are proprietary and therefore somewhat opaque means that you should carefully evaluate them for your specific use case before committing to ongoing licensing charges.

Links to previous benchmarks

If you’re interested in more details of around the previous benchmarks.

Links to getting started guides

If you’re interested in getting started with any of the accelerator hardware I used during my first benchmark I’ve put together getting started guides for the Google, Intel, and NVIDIA hardware I looked at during that analysis.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alasdair Allan

Alasdair Allan

Scientist, Author, Hacker, Maker, and Journalist.