Machine learning is one of the hottest technologies of the last decade. You may not even realize it’s everywhere.

Applications such as augmented reality, self-driving vehicles, chatbots, computer vision, social media, among others, have adopted machine learning technology to solve problems.

The good news is that numerous machine-learning resources and frameworks are available to the public. Two of those are TensorFlow and Teachable Machine.

In this Flutter tutorial, you’ll develop an application called Plant Recognizer that uses machine learning to recognize plants simply by looking at photos of them. You’ll accomplish this by using the Teachable Machine platform, TensorFlow Lite, and a Flutter package named tflite_flutter.

By the end of this tutorial, you’ll learn how to:

• Use machine learning in a mobile app.
• Train a model using Teachable Machine.
• Integrate and use TensorFlow Lite with the tflite_flutter package.
• Build a mobile app to recognize plants by image.

TensorFlow is a popular machine-learning library for developers who want to build learning models for their apps. TensorFlow Lite is a mobile version of TensorFlow for deploying models on mobile devices. And Teachable Machine is a beginner-friendly platform for training machine learning models.

Getting Started

Download the project by clicking Download Materials at the top or bottom of the tutorial and extract it to a suitable location.

After decompressing, you’ll see the following folders:

1. final: contains code for the completed project.
2. samples: has sample images you can use to train your model.
3. samples-test: houses samples you can use to test the app after it’s completed.
4. starter: the starter project. You’ll work with this in the tutorial.

Open the starter project in VS Code. Note that you can use Android Studio, but you’ll have to adapt the instructions on your own.

VS Code should prompt you to get dependencies — click the button to get them. You can also run flutter pub get from the terminal to get the dependencies.

Build and run after installing the dependencies. You should see the following screen:

The project already allows you to pick an image from the camera or media library. Tap Pick from gallery to select a photo.

As you can see, the app doesn’t recognize images. You’ll use TensorFlow Lite to solve that in the next sections. But first, here’s an optional, high-level overview of machine learning to give you a gist of what you’ll do.

Brief Introduction to Machine Learning

This section is optional because the starter project contains a trained model model_unquant.tflite and classification labels in the labels.txt file.

If you’d prefer to dive into TensorFlow Lite integration, feel free to skip to Installing TensorFlow Lite.

What is Machine Learning

In this Flutter tutorial, you need to solve a classification problem: plant recognition. In a traditional approach, you’d define rules to determine which images belong to which class.

The rules would be based on patterns such as that of a sunflower, which has a large circle in the center, or a rose, which is somewhat like a paper ball. It goes like the following:

The traditional approach has several problems:

• There are many rules to set when there are many classification labels.
• It’s subjective.
• Some rules are hard to determine by the program. For example, the rule “like a paper ball” can’t be determined by a program because a computer doesn’t know what a paper ball looks like.

Machine learning offers another way to solve the problem. Instead of you defining the rules, the machine defines its own rules based on input data you provide:

The machine learns from the data, and that’s why this approach is called machine learning.

Before you continue, here’s some terminology you may need to know:

Training: The process by which the computer learns data and derives rules.

Model: The object created from training. It comprises the algorithm used to solve the AI problem and the learned rules.

Building a Model with Teachable Machine

Now, you’ll learn how to train a model with Teachable Machine. The steps you’ll follow include:

1. Preparing the dataset
2. Training the model
3. Exporting the model

Your first step is to prepare your dataset — the project needs plant photos. So your dataset is a collection of plants you want to recognize.

Preparing the Dataset

In a production-ready app, you’d want to collect as many varieties of a plant and as many plants as possible for your dataset to ensure higher accuracy. You’d do that by using your phone camera to take pictures of these plants or download images from various online sources that offer free datasets such as this one from Kaggle.

However, this tutorial uses plants from the samples folder, so you can also use it as a starting point.

Whichever one you use, it’s important to keep the number of samples for each label at similar levels to avoid introducing bias to the model.

Training the Model

Next, you’ll learn how to train the model using Teachable Machine.

First, go to https://teachablemachine.withgoogle.com and click Get Started to open the training tool:

Then select Image Project:

Choose Standard Image Model, because you’re not training a model to run on a microcontroller:

Once you’ve entered the training tool, add the classes and edit the labels of each class, as shown below:

Next, add your training samples by clicking Upload under each class. Then, drag the folder of the appropriate plant type from the samples folder to the Choose images from your files … panel.

After you’ve added all the training samples, click Train Model to train the model:

After the training completes, test the model with other plant images.

Use the images in the samples-test folder, like so:

Finally, export the model by clicking Export Model on the Preview panel. A dialog displays:

In the dialog, choose TensorFlow Lite. That’s because your target platform is mobile.

Next, select Floating point conversion type for the best predictive performance. Then, click Download my model to convert and download the model.

It may take several minutes to complete the model conversion process. Once it’s done, the model file will automatically download to your system.

After you have the model file converted_tflite.zip in hand, decompress it and copy labels.txt and model_unquant.tflite to the ./assets folder in your starter project.

Here’s what each of those files contains:

• labels.txt: The label of each class.
• model_unquant.tflite: The trained machine learning model for plant recognition.

Training a Model: How it Works

TensorFlow uses an approach called deep learning, which is a subset of machine learning. Deep learning utilizes a network structure with many layers, similar to what’s shown below:

To explain it further:

• The input data feeds into the first layer: If the input data is an image, the pixels of the image feed into the first layer.
• The output result is stored in the last layer: If the network is solving a classification problem, this layer stores the possibility of each class.
• The layers in between are called hidden layers. They contain formulas with parameters that sit in the node. The input values flow to those layers, which ultimately calculate the final results.

Deep learning tunes the parameters in the hidden layers to achieve prediction results that are the same as the provided result. Many iterations are required for the machine-training process to achieve well-tuned parameters.

Every iteration includes the following actions:

• Run the prediction step using the input sample.
• Compare the prediction result against the provided result. The system will calculate how much difference between them, and this value is called loss.
• Modify the parameters in the hidden layers to minimize loss.

After the iterations are complete, you’ll have optimized parameters, and your results will have the highest possible precision.

Understanding Tensor and TensorFlow Prediction

For the training and prediction process, TensorFlow uses a data structure called Tensors as the input and output — hence why Tensor is a part of the name TensorFlow.

A Tensor is a multidimensional array that represents your input data and the machine-learning result.

The following definitions may help you understand what a tensor is, relative to what you already know:

• Scalar: Single value, for example: 1, 2, 3.3
• Vector: Multiple-axis value, examples: (0, 0), (1, 2, 3)
• Tensor: Multiple-dimension value. Example is: (((0, 0), (1, 0)), ((1,1), (2,2)))

In an image classification problem, the input tensor is an array that represents an image, similar to this:

[ 
 // First line of the first image 
 [
   // First Pixel of the first line 
   [0.0, 0.0, 1.0], 
   // Second Pixel of the first line 
   [0.0, 0.0, 1.0], 
   [1.0, 1.0, 0.0], ...
 ]
 // Second line of the first image 
 ... 
 ... 
]

To explain further:

• The first layer of the array represents every line of the image.
• The second layer of the array represents every pixel of the line.
• The last layer represents the color of the pixel, which is red, green, or blue.

If you resample the image to 200×200, the shape of the tensor is [200, 200, 3].

The output tensor is an array of the score for each label, for example:
[0.1, 0.8, 0.1, 0]. In this case, each value corresponds to a label, for example, rose, tulip, sunflower and daisy.

Notice that in the example, the value for the tulip label is 0.8 — meaning the possibility that the image shows a tulip is 80%, the others are 10% and daisy 0%. The shape of the output here is [4].

The following diagram further illustrates the data flow:

Since TensorFlow uses tensors for the inputs and outputs, you need to do preprocessing so that TensorFlow understands the input data and postprocessing so that human users can understand the output data. You’ll install TensorFlow Lite in the next section to process the data.

Installing TensorFlow Lite in Flutter

To use TensorFlow in your Flutter app, you need to install the following packages:

• tflite_flutter: allows you to access the native TensorFlow Lite library. When you invoke the methods of tflite_flutter, it calls the corresponding method of the native TensorFlow Lite SDK.
• tflite_flutter_helper: enables you to manipulate TensorFlow inputs and outputs. For example, it converts image data to tensor structure. It reduces the effort required to create pre- and post-processing logic for your model.

Open pubspec.yaml and add them in the dependencies section:

tflite_flutter: ^0.9.0
tflite_flutter_helper: ^0.3.1

Then, run flutter pub get to get packages.

tflite_flutter_helper:
 git:
  url: https://github.com/filofan1/tflite_flutter_helper.git
  ref: 783f15e5a87126159147d8ea30b98eea9207ac70

Get packages again.

Then, if you are building for Android, run the installation script below on macOS/Linux:

./install.sh 

If you’re on Windows, run install.bat instead:

install.bat 

However, to build for iOS, you need to download TensorFlowLiteC.framework, decompress it and place TensorFlowLiteC.framework in the .pub-cache folder for tflite_flutter. The folder location is /home/USER/.pub-cache/hosted/pub.dartlang.org/tflite_flutter-0.9.0/ios/, where USER is your username. If you’re not using version 0.9.0, place it at the corresponding version.

You’re just adding dynamic Android and iOS libraries so that you can run TensorFlow Lite on your target platform.

Creating an Image Classifier

In machine learning, classification refers to predicting the class of an object out of a finite number of classes, given some input.

The Classifier included in the starter project is a skeleton of the image classifier that you’ll create to predict the category of a given plant.

By the end of this section, the Classifier will be responsible for these steps:

1. Load labels and model
2. Preprocess image
3. Use the model
4. Postprocess the TensorFlow output
5. Select and build the category output

Your initialization code will load the labels and the model from your files. Then, it’ll build TensorFlow structures and prepare them to be used by a call to predict().

Your prediction action will include several parts. First, it’ll convert a Flutter image to a TensorFlow input tensor. Then, it’ll run the model and convert the output to the final selected category record that contains the label and score.

Importing the Model to Flutter

There are two pieces of data that you’ll load into the program: the machine learning model – model_unquant.tflite and the classification labels — labels.txt, which you got from the Teachable Machine platform.

To begin, make sure to include the assets folder in pubspec.yaml:

assets:
  - assets/

The assets record is responsible for copying your resource files to the final application bundle.

Loading Classification Labels

Open lib/classifier/classifier.dart and import tflite_flutter_helper:

import 'package:tflite_flutter_helper/tflite_flutter_helper.dart';

Then add the following code after predict:

static Future<ClassifierLabels> _loadLabels(String labelsFileName) async {
  // #1
  final rawLabels = await FileUtil.loadLabels(labelsFileName);

  // #2
  final labels = rawLabels
    .map((label) => label.substring(label.indexOf(' ')).trim())
    .toList();

  debugPrint('Labels: $labels');
  return labels;
}

Here’s what the above code does:

1. Loads the labels using the file utility from tflite_flutter_helper.
2. Removes the index number prefix from the labels you previously downloaded. For example, it changes 0 Rose to Rose.

Next, replace // TODO: _loadLabels in loadWith by calling _loadLabels like so:

final labels = await _loadLabels(labelsFileName);

This code loads the label file.

Save the changes. There is nothing more to do with the labels now, so it’s time to run a test.

Build and run.

Look at the console output:

Congrats, you successfully parsed the model’s labels!

Importing TensorFlow Lite Model

Go to lib/classifier/classifier_model.dart and replace the contents with the following code:

import 'package:tflite_flutter/tflite_flutter.dart';

class ClassifierModel {
 Interpreter interpreter;

 List<int> inputShape;
 List<int> outputShape;

 TfLiteType inputType;
 TfLiteType outputType;

 ClassifierModel({
  required this.interpreter,
  required this.inputShape,
  required this.outputShape,
  required this.inputType,
  required this.outputType,
 });
}

ClassifierModel stores all model-related data for your classifier. You’ll use the interpreter to predict the results. inputShape and outputShape are shapes for the input and output data respectively while inputType and outputType are the data types of the input and output tensors.

Now, import the model from the file. Go to lib/classifier/classifier.dart and add the following code after _loadLabels:

static Future<ClassifierModel> _loadModel(String modelFileName) async {
  // #1
  final interpreter = await Interpreter.fromAsset(modelFileName);

  // #2
  final inputShape = interpreter.getInputTensor(0).shape;
  final outputShape = interpreter.getOutputTensor(0).shape;

  debugPrint('Input shape: $inputShape');
  debugPrint('Output shape: $outputShape');

  // #3
  final inputType = interpreter.getInputTensor(0).type;
  final outputType = interpreter.getOutputTensor(0).type;

  debugPrint('Input type: $inputType');
  debugPrint('Output type: $outputType');
  
  return ClassifierModel(
   interpreter: interpreter,
   inputShape: inputShape,
   outputShape: outputShape,
   inputType: inputType,
   outputType: outputType,
  );
 }

Don’t forget to add the import import 'package:tflite_flutter/tflite_flutter.dart'; at the top.

Here’s what happens in the above code:

1. Creates an interpreter with the provided model file — the interpreter is a tool to predict the result.
2. Read the input and output shapes, which you’ll use to conduct pre-processing and post-processing of your data.
3. Read the input and output types so that you’ll know what type of data you have.

Next, replace // TODO: _loadModel in loadWith with the following:

final model = await _loadModel(modelFileName);

The code above loads the model file.

Build and run. Look at the console output:

You successfully parsed the model! It’s a multi-dimensional array of float32 values.

Finally, for initialization, replace // TODO: build and return Classifier in loadWith with the following:

return Classifier._(labels: labels, model: model);

That builds your Classifier instance, which PlantRecogniser uses to recognize images the user provides.

Implementing TensorFlow Prediction

Before doing any prediction, you need to prepare the input.

You’ll write a method to convert the Flutter Image object to TensorImage, the tensor structure used by TensorFlow for images. You also need to modify the image to fit the required shape of the model.

Pre-Processing Image Data

With the help of tflite_flutter_helper, image processing is simple because the library provides several functions you can pull in to handle image reshaping.

Add the _preProcessInput method to lib/classifier/classifier.dart:

TensorImage _preProcessInput(Image image) {
  // #1
  final inputTensor = TensorImage(_model.inputType);
  inputTensor.loadImage(image);

  // #2
  final minLength = min(inputTensor.height, inputTensor.width);
  final cropOp = ResizeWithCropOrPadOp(minLength, minLength);

  // #3
  final shapeLength = _model.inputShape[1];
  final resizeOp = ResizeOp(shapeLength, shapeLength, ResizeMethod.BILINEAR);

  // #4
  final normalizeOp = NormalizeOp(127.5, 127.5);

  // #5
  final imageProcessor = ImageProcessorBuilder()
    .add(cropOp)
    .add(resizeOp)
    .add(normalizeOp)
    .build();

  imageProcessor.process(inputTensor);

  // #6
  return inputTensor;
 }

_preProcessInput preprocesses the Image object so that it becomes the required TensorImage. These are the steps involved:

1. Create the TensorImage and load the image data to it.
2. Crop the image to a square shape. You have to import dart:math at the top to use the min function.
3. Resize the image operation to fit the shape requirements of the model.
4. Normalize the value of the data. Argument 127.5 is selected because of your trained model’s parameters. You want to convert image’s pixel 0-255 value to -1...1 range.
5. Create the image processor with the defined operation and preprocess the image.
6. Return the preprocessed image.

Then, invoke the method inside predict(...) at // TODO: _preProcessInput:

final inputImage = _preProcessInput(image);

debugPrint(
 'Pre-processed image: ${inputImage.width}x${image.height}, '
 'size: ${inputImage.buffer.lengthInBytes} bytes',
);

You’ve implemented your pre-processing logic.

Build and run.

Pick an image from the gallery and look at the console:

You successfully converted the image to the model’s required shape!
Next, you’ll run the prediction.

Running the Prediction

Add the following code at // TODO: run TF Lite to run the prediction:

// #1
final outputBuffer = TensorBuffer.createFixedSize(
 _model.outputShape,
 _model.outputType,
);

// #2
_model.interpreter.run(inputImage.buffer, outputBuffer.buffer);
debugPrint('OutputBuffer: ${outputBuffer.getDoubleList()}');

Here’s what happens in the code above:

1. TensorBuffer stores the final scores of your prediction in raw format.
2. Interpreter reads the tensor image and stores the output in the buffer.

Build and run.

Select an image from your gallery and observe the console:

Great job! You successfully got an interpretive result from the model. Just a few more steps to make the results friendly for human users. That brings you to the next task: post-processing the result.

Post-Processing the Output Result

The TensorFlow output result is a similarity score for each label, and it looks like this:

[0.0, 0.2, 0.9, 0.0]

It’s a little hard to tell which value refers to which label unless you happened to create the model.

Add the following method to lib/classifier/classifier.dart:

List<ClassifierCategory> _postProcessOutput(TensorBuffer outputBuffer) {
  // #1
  final probabilityProcessor = TensorProcessorBuilder().build();

  probabilityProcessor.process(outputBuffer);

  // #2
  final labelledResult = TensorLabel.fromList(_labels, outputBuffer);

  // #3
  final categoryList = <ClassifierCategory>[];
  labelledResult.getMapWithFloatValue().forEach((key, value) {
   final category = ClassifierCategory(key, value);
   categoryList.add(category);
   debugPrint('label: ${category.label}, score: ${category.score}');
  });
   
  // #4
  categoryList.sort((a, b) => (b.score > a.score ? 1 : -1));

  return categoryList;
 }

Here’s the logic for your new post-processing method:

1. Create an instance of TensorProcessorBuilder to parse and process the output.
2. Map output values to your labels.
3. Build category instances with the list of label – score records.
4. Sort the list to place the most likely result at the top.

Great, now you just need to invoke _postProcessOutput() for the prediction.

Update predict(...) so that it looks like the following:

ClassifierCategory predict(Image image) {
 // Load the image and convert it to TensorImage for TensorFlow Input
 final inputImage = _preProcessInput(image);

 // Define the output buffer
 final outputBuffer = TensorBuffer.createFixedSize(
  _model.outputShape,
  _model.outputType,
 );

 // Run inference
 _model.interpreter.run(inputImage.buffer, outputBuffer.buffer);

 // Post Process the outputBuffer
 final resultCategories = _postProcessOutput(outputBuffer);
 final topResult = resultCategories.first;

 debugPrint('Top category: $topResult');

 return topResult;
}

You implemented your new post-processing method in your TensorFlow output, so you get the first and most valuable result back.

Build and run.

Upload an image and see it correctly predicts the plant:

Congratulations! That was a good ride.
Next, you’ll learn how the Classifier works to produce this result.

Using the Classifier

Now that it’s built, you’d probably like to understand how this app uses Classifier to determine the name of the plant and display the results.

All the code from this section is already implemented in the starter project, so just read and enjoy!

Picking an Image From the Device

Your machine needs a photo to analyze, and you need to allow users to capture a photo they took from either the camera or photo album.

This is how you do that:

void _onPickPhoto(ImageSource source) async {
  // #1 
  final pickedFile = await picker.pickImage(source: source);

  // #2 
  if (pickedFile == null) {
   return;
  }

  // #3 
  final imageFile = File(pickedFile.path);

  // #4 
  setState(() {
   _selectedImageFile = imageFile;
  });
   
   
 }

And here’s how the code above works:

1. Pick an image from the image source, either the camera or photo album.
2. Implement handling in case the user decides to cancel.
3. Wrap the selected file path with a File object.
4. Change the state of _selectedImageFile to display the photo.

Initializing the Classifier

Here’s the code used to initialize the classifier:

@override
void initState() {
 super.initState();
 // #1
 _loadClassifier();
}

Future _loadClassifier() async {
 debugPrint(
  'Start loading of Classifier with '
  'labels at $_labelsFileName, '
  'model at $_modelFileName',
 );

 // #2
 final classifier = await Classifier.loadWith(
  labelsFileName: _labelsFileName,
  modelFileName: _modelFileName,
 );
  
 // #3
 _classifier = classifier;
}

Here’s how that works:

1. Run asynchronous loading of the classifier instance. Note that the project doesn’t contain enough error-handling code for production, so the app may crash if something goes wrong.
2. Call loadWith(...) with the file paths for your label and model files.
3. Save the instance to the widget’s state property.

Analyzing Images Using the Classifier

Look at the following code in PlantRecogniser at lib/widget/plant_recogniser.dart.

void _analyzeImage(File image) async {

  // #1 
  final image = img.decodeImage(image.readAsBytesSync())!;

  // #2 
  final resultCategory = await _classifier.predict(image);

  // #3 
  final result = resultCategory.score >= 0.8
    ? _ResultStatus.found
    : _ResultStatus.notFound;
   
  // #4 
  setState(() {
   _resultStatus = result;
   _plantLabel = resultCategory.label;
   _accuracy = resultCategory.score * 100;
  });
 }

The above logic works like this:

1. Get the image from the file input.
2. Use Classifier to predict the best category.
3. Define the result of the prediction. If the score is too low, less than 80%, it treats the result as Not Found.
4. Change the state of the data responsible for the result display. Convert the score to a percentage by multiplying it by 100.

You then invoked this method in _onPickPhoto() after imageFile = File(pickedFile.path);:

void _onPickPhoto(ImageSource source) async {
  ...
  final imageFile = File(pickedFile.path);
  _analyzeImage(imageFile);
}

Here’s the effect when everything is set:

Where to Go from Here?

Great job. You made it to the end of this TensorFlow and Flutter tutorial!

Download the completed project by clicking Download Materials at the top or bottom of the tutorial.

You learned how to use TensorFlow Lite in a Flutter application, and if you weren’t familiar with machine learning already — you are now.

You also have the basic skills needed to implement a machine-learning solution that can solve problems and answer questions for your users.

If you’re interested in exploring classification more deeply, check out our Machine Learning: End-to-end Classification tutorial to learn more.

Also, if you’d like to learn more about normalizing data for a model from TensorFlow’s documentation, take a look at TensorFlow’s normalization and quantization parameters.

Furthermore, if you need a more robust solution than you can create with Teachable Machine, you could use a different learning framework such as Keras or PyTorch to create machine-learning models. These frameworks are more difficult to learn than Teachable Machine; however, they provide more features.

Tensorflow and Teachable Machine have quite a bit more to offer, and here is the best place to go to learn:

We hope you have enjoyed this tutorial. If you have any questions or comments, please join the forum discussion below!

Ken Lee

Over 300 content creators. Join our team.