Benchmarking TensorFlow and TensorFlow Lite on the Raspberry Pi

Headline results from benchmarking

Part I—Benchmarking

A more detailed analysis of the results

Benchmarking results in milli-seconds for MobileNet v1 SSD 0.75 depth model and the MobileNet v2 SSD model both trained using the Common Objects in Context (COCO) dataset with an input size of 300×300.
Inferencing speeds in milli-seconds for MobileNet SSD V1 (blue) and MobileNet SSD V2 (green) across all tested platforms. Lower numbers are good!

Heating and Cooling

Peak external and peak CPU temperatures during inferencing in °C.
Measuring the external temperature of the Raspberry Pi.
$ paste <(cat /sys/class/thermal/thermal_zone*/type) <(cat /sys/class/thermal/thermal_zone*/temp) | column -s $'\t' -t | sed 's/\(.\)..$/.\1°C/'
cpu-thermal 77.3°C
Peak external (red, left hand bars) and peak CPU (purple, right hand bars) temps during inferencing in °C.

Summary

Part II—Methodology

Installing TensorFlow Lite on the Raspberry Pi

Everything you need to get started setting up the Raspberry Pi
% ssh pi@raspberrypi.local
$ sudo apt-get update
$ sudo apt-get install build-essential
$ sudo apt-get install git
$ sudo apt-get install libatlas-base-dev
$ sudo apt-get install python3-pip
$ git clone https://github.com/PINTO0309/Tensorflow-bin.git
$ cd Tensorflow-bin
$ pip3 install tensorflow-1.13.1-cp35-cp35m-linux_armv7l.whl
$ python3 -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
$ sudo apt-get install libwebp6 libwebp-dev
$ sudo apt-get install libtiff5 libtiff5-dev
$ sudo apt-get install libjasper1 libjasper-dev
$ sudo apt-get install libilmbase12 libilmbase-dev
$ sudo apt-get install libopenexr22 libopenexr-dev
$ sudo apt-get install libgstreamer0.10-0 libgstreamer0.10-dev
$ sudo apt-get install libgstreamer1.0-0 libgstreamer1.0-dev
$ sudo apt-get install libavcodec-dev
$ sudo apt-get install libavformat57 libavformat-dev
$ sudo apt-get install libswscale4 libswscale-dev
$ sudo apt-get install libqtgui4
$ sudo apt-get install libqt4-test
$ pip3 install opencv-python
$ pip3 install Pillow
$ pip3 install numpy

The benchmarking code

Converting models to TensorFlow Lite format

$ cd ~
$ wget http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tar.gz
$ tar -zxvf ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tar.gz
$ bazel build tensorflow/tools/graph_transforms:summarize_graph
$ bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=/Users/aa/Downloads/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18/tflite_graph.pb
Found 1 possible inputs: (name=normalized_input_image_tensor, type=float(1), shape=[1,300,300,3])No variables spotted.Found 1 possible outputs: (name=TFLite_Detection_PostProcess, op=TFLite_Detection_PostProcess)Found 4137705 (4.14M) const parameters, 0 (0) variable parameters, and 0 control_edgesOp types used: 451 Const, 389 Identity, 105 Mul, 94 FakeQuantWithMinMaxVars, 70 Add, 35 Sub, 35 Relu6, 35 Rsqrt, 34 Conv2D, 25 Reshape, 13 DepthwiseConv2dNative, 12 BiasAdd, 2 ConcatV2, 1 RealDiv, 1 Sigmoid, 1 Squeeze, 1 Placeholder, 1 TFLite_Detection_PostProcess
$ bazel run tensorflow/lite/toco:toco -- --input_file=/Users/aa/Downloads/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18/tflite_graph.pb --output_file=/Users/aa/Downloads/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18/tflite_graph.tflite --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' --inference_type=QUANTIZED_UINT8 --mean_values=128 --std_values=128 --change_concat_input_ranges=false --allow_custom_ops

In closing

Links to getting started guides

--

--

--

Scientist, Author, Hacker, Maker, and Journalist.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Predict House Prices using Python

Let’s build an object detector using Python + Machine Learning!

Developing deep learning model to generate clinical reports for X-ray images

Create an AI to Play Chrome Dino Game with CNN

How to Build a Predictive Machine Learning Site With React and Python

Classifying Clinically Actionable Genetic Mutations—Re-Examining Past Kaggle Competition with…

Review: Dynamic Routing Between Capsules

Reinforcement Learning for Marketing: Lessons and Challenges

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alasdair Allan

Alasdair Allan

Scientist, Author, Hacker, Maker, and Journalist.

More from Medium

AI Pytorch HW1 answer, tutorial and thought.

How to Install and configure Qt5 manually on Ubuntu 18.04

Capture Pytorch concept for better understanding

Machine Learning on embedded devices