Neural Networks on iOS and Android: Classify Images with TensorFlow Lite

Neural Networks on Mobile Devices with TensorFlow Lite: A Tutorial

Training on your own Data

This will be a practical, end-to-end guide on how to build a mobile application using TensorFlow Lite that classifies images from a dataset for your projects.

This application uses live camera and classifies objects instantly. The TFLite application will be smaller, faster, and more accurate than an application made using TensorFlow Mobile, because TFLite is made specifically to run neural nets on mobile platforms.

We’ll be using the MobileNet model to train our network, which will keep the app smaller.

Getting Started

Requirements…

1. Python 3 .5 or higher — python3 -V
2. Tensorflow 1.9 or higher — pip3 install — upgrade tensorflow

Also, open the terminal and type:

alias python=python3

Now, python3 will open with the python command. This will make it easier to implement the code just by copy-pasting without having to worry about 3 after typing Python.

The TFLite tutorial contains the following steps:

Step 1: Download the Code Files

Let’s start by downloading the code from the tensorflow-for-poets GitHub. Open the command prompt where you want to download the folder and type:

This will download the files and make a new folder called tensorflow-for-poets in your current directory.

Output

Download GitHub Files

FYI : You can change the name of the folder to your project name after downloading.

Info: The folder contains the sub-folders

scripts — Contains the machine learning code .py files.

tf_files — It will contain output files like models — graph.pb, labels.txt

android — Contains Android app projects for both tfmobile and TFlite.

iOS — Contains the iOS app project files using x Code.

Step 2: Download the Dataset

Let’s download a 200MB publicly available dataset with 5 different flowers to classify from. Then extract the flower_photos.tgz inside the tf_files folder which will look something like this:

<em>tensorflow-for-poets-2 > tf_files > flower_photos</em>

Info — The 5 different category folders are Rose, Daisy, Dandelion, Sunflower and Tulip.

Step 3: Retrain the model

Open the command prompt inside the “tensorflow-for-poets-2” folder and type:

Output

This will download the pre-trained frozen graph mobilenet_1.0_244

Download Graph

and create retrained_graph.pb and retrained_labels.txt files in the tf_files folder.

Completed Result

Open Tensorboard

Open another command prompt in the current directory and point tensorboard to the summaries_dir :

Output

Open Tensorboard

Now you can open the 6006 port in your browser to see the results.

Visualization

:red_circle: Training & :large_blue_circle: Validation

Tensorboard | Cross Entropy/Loss (Left) | Accuracy (Right)

Accuracy is above 0.90 and loss is below 0.4 for validation.

Verify :heavy_check_mark:

I downloaded a random rose image from the Internet in the current folder and used the following command to run the label_image.py script to detect the --image using the --graph file.

Output

The result gives 99% accuracy for the new_rose image.

Step 4: Convert the Model to TFLite Format

Toco is used to convert the file to .lite format. For more detail on toco arguments, use toco --help

This will create an optimized_graph.lite file in your tf_files directory.

Info : Use the toco -h for more details

--input_file has been updated to --graph_def_file

--input_format is not need for the mobile_net graphs.

Sign in with your Apple ID - Apple Developer

Use the Apple ID you used to register or register now. developer.apple.com

:lollipop: :icecream: Android :bread::bee: :doughnut:

Step 5: Setup Android Studio and Test Run

There are two ways to do this: Android Studio and Bazel. I’ll be using AS since more people are familiar with it.

If you don’t have it installed already, go here and install it:

Test Run :runner:

To make sure everything is working correctly in Android Studio, let’s do a test run.

:small_orange_diamond: Open Android Studio and select “:file_folder:Open an existing Android Studio project”.

:small_orange_diamond: Go to android/tfmobile directory.

:small_orange_diamond: If everything works perfectly, click the BUILD>BUILD APK button.

Open the folder containing app-debug.apk file by clicking locate .

Note: Turn On the developer mode in your phone before installing the app.

Step 6: Run your Application

First move the trained files into the assets folder of the application .

Replace the graph.lite file.

and then labels.txt file.

Now click Tools>> Build .apk file.

Install Application

Install the .apk file onyour phone and see the re-trained neural network detecting the objects.

:clap: :clap: :clap: Congratulations! Now you can apply the same method in your next gazillion dollar app, enable doctors to work faster and better without expensive equipment in rural parts of the world, or just have fun. :clap::clap: :clap:

If you hit a wall while implementing this post, reach out in the comments.

For further tutorials on how to use TensorFlow in mobile apps follow me on Medium and Twitter to see similar posts.

Clap it! Share it! Follow Me!