Building Vision-Controlled Car using Raspberry Pi

This tutorial creates a car that is controlled using a camera. A wood plank is used as the car body which holds all car components. The components are 1 USB camera, 1 Raspberry Pi (RPi), 1 mini electronic breadboard (BB), 4 DC toy motors, 4 car tires, and more. After fixing the component using cable ties, the RPi will get connected to the motors using its general purpose input output (GPIO) pins. Similar to a traffic sign, the motors run when the dominant color in the captured images is green and stop when it is red.

Cover image by Ahmed Fawzy Gad. Building Vision-Controlled Car using Raspberry Pi

The tutorial has 2 experiments. The first one creates a car that moves without using the camera. This is just for testing that everything works well and making things simpler. The second experiment uses the camera which will be placed on the car for controlling its movement.

Before starting the tutorial, it is preferred to have a beginners-level knowledge about electronics and be able to create a circuit for lighting a led using RPi. Also knowledge about how transistor and diode work is essential.

The steps of this tutorial are as follows:

Preparing the Car Components
Fixing the Components on the Wood Plank
Understanding the Circuit Connected to the Motor
Building the Circuit on the BB using Transistors, Resistors, and Diodes
Connecting the BB to the RPi GPIO Pins
Accessing RPi
Controlling Car Motors
Connecting a USB Camera to RPi
Controlling the Motors using the Camera

Let’s start discussing each of these steps.

1. Preparing the Car Components

There are 12 different components required. Multiple pieces may be required from the same component. The summary is as follows:

1 RPi : Processes the captured images and generating signals over its GPIO output pins for running or stopping the motors.

2. 1 Mini Electronic BB: It holds the circuits connecting the motors to the RPi pins.

3. 4 DC Toy Motors: Such motors push the car forward.

4. 1 Wood Plank: Components in point 2, 3, and 4 will be placed on this plank. The plank is labeled according to the position of each component. M1, M2, M3, and M4 refers to the 4 motors.

5. 6 Plastic Cable Ties: Used for fixing the previous components (1 RPi, 1 BB, and 4 DC motors) on the wood plank. 6 cables are needed because there are 6 pieces to be fixed. Note that each component has two holes to their sides for passing the cable tie around the component. The holes for the BB are bounded in red in the wood plank image.

6. 4 Car Tires: Each tire is connected to one motor. Make sure that the tire fits the axis in the motor.

7. 13 Male/Male Jumper Wires: Such wires connect the components placed on the BB.

8. 3 Male/Female Jumper Wires: The female end of such wires is connected to the GPIO pins of the RPi. The male end is connected to the BB.

9. 2 NPN MPS2222A Transistors: The output current from the RPi GPIO pins does not allow the motor to work. Transistor amplifies such current. As an alternative to the MPS2222A, you could use 2N2222A.

10. 2 1N4148 Diodes: Used for passing the current in one direction. If the amplified signal produced by the transistor returned back to the GPIO pins, the pins will be damaged.

11. 2 1K Resistors: Connects the transistor base terminal to the GPIO output pin.

12. 1 USB Camera: Used for capturing images and controlling the movement of the car. Based on the color in such images, the car motors may run or stop. It will be used in the second experiment.

Note that there are just 2 transistors, 2 resistors, and 2 diodes. Every single transistor-resistor-diode combination builds a single circuit connected to a single motor. In this tutorial, just 2 motors will be used for pushing the car and thus just 2 circuits required. This is why there are just 2 pieces for the transistor, resistor, and diode.

After placing all components required for building the body of the car on the wood plank according to the following figure, next is to use the cable tie for fixing them.

2. Fixing the Components on the Wood Plank

The following figure shows the front side of the wood plank after fixing the components using the cable ties. Make sure that no component is shaking, especially the motors. It is also very important to use car tires that lift up the wood blank from the ground so that the back side of the cable ties are not touching the ground. This affects the movement of the car. Take care of not making the cable to touch anything in the RPi to avoid damaging its elements.

After making sure that the car elements are placed on the wood plank correctly, next is to build the circuits that will be connected to both the motors and the RPi GPIO pins.

3. Understanding the Circuit Connected to the Motor

The BB will hold two circuits, each circuit connected to a single motor. The circuit diagram is given below. The circuit uses a transistor, a resistor, and a diode. Let’s understand the benefit of this circuit and how such components are connected to each other.

The maximum current produced by a GPIO pin in the RPi is around 16mA. This current is not sufficient for running the DC toy motor. Thus we have to increase that current. This is done using the transistor which amplifies the current.

The transistors have three legs which are emitter, base, and collector. The base is the middle leg and the other legs are on the sides of the transistor. It is important to know the label of each leg in order to build the circuit successfully. You search Google for more info ( http://www.google.com/search?q=npn+transistor+polarity+multimeter ).

The transistor emitter is connected to the ground (GND) RPi GPIO pin. The transistor base is connected to a 1K resistor which is then connected to the RPi GPIO output pin.

The transistor collector is connected to the anode of the diode. The diode cathode is connected to the 5 voltage pin in the RPi GPIO. Some diodes have a bold black circle near to the cathode terminal. This helps to know the 2 terminals of the diode. You can also use the multimeter device to check the polarity of the diode. When using the multimeter in the mega resistor mode while connecting the multimeter terminals to the diode. When the device reads a value for the resistance, this means that the positive terminal of the device touches the anode of the diode and the negative terminal of the device touches the cathode.

The motor is connected in parallel to the diode. When the state of the GPIO output pin connected to the transistor is high, the NPN transistor will be active and current passes from the emitter to the collector. The diode will be activated and thus the motor runs. The diode prevents the current to flow in the reverse direction. If the amplified signal from the transistor flows back to the GPIO pin of the RPi, the pin will be damaged.

Note that reversing the connection of the motor will make it still work but in the reverse direction. This helps to build a car that moves forward and backward.

After understanding the circuit, let’s build them in the BB.

4. Building the Circuit on the BB using Transistors, Resistors, and Diodes

The connections between the three components (transistor, resistor, and diode) on the BB are illustrated in the following figure. Remember to connect the correct terminals of the transistors and the diodes to each other. Let’s explore the connections for the circuit at the top of the figure. This circuit is connected to the motor labeled M2 in the wood plank.

The resistor is connected to the pins in columns E and F in row 4. The base of the transistor is connected to the pin in column G and row 4. Because columns F and G are connected to each other in the breadboard, then both the resistor and the base are connected to each other.

The anode of the diode is connected to the pin in column I and row 5. Similarly, the cathode of the diode is connected to the pin in column I but row 8. Note that the collector of the transistor is connected to the pin in row 5 and column G. Because columns I and G are connected to each other in the breadboard, then the collector of the transistor is connected to the anode of the diode.

According to the wood plank labels, the motor labeled M2 is connected in parallel with the diode using its pins in column I and row 5 and 8.

The emitter of the transistor is connected to the pin in row 4 and column G. According to the previous diagram, the emitter is connected to the GND pin of the RPi GPIO. This will be shown later. There are other 2 connections to the RPi GPIO pins. The first one is between the cathode of the diode and a 5V GPIO pin. The second one is the connection between the terminal of the resistor in column E and row 4 and one GPIO output pins.

The previous connections will be repeated for the second circuit of the motor labeled M1 in the wood plank. As in the first circuit, there will be 3 missing connections to the RPi GPIO pins. Let’s show the connections between the BB to the RPi GPIO pins. Up to this time, all jumper wires used are male/male.

5. Connecting the BB to the RPi GPIO Pins

There are three connections required between each circuit to the RPi GPIO pins. The first one connects one of the GND pins of RPi to the emitter of the NPN transistor. The second connection connects the cathode of the diode to one of the 5V pins in the RPi. The third connection connects the resistor to one of the GPIO output pins.

The following diagram shows how the circuits created previously in the BB are connected to the GPIO pins of the RPi. According to the board numbering of the GPIO pins, pin number 4 is used for 5V, pin number 6 as a GND, and pin number 8 as the output pin. The 5V pin is connected to the BB using the pin in column J and row 28. From the same row in the BB, two connections will connect the cathode of the diode to the 5V pin of RPi.

The GND pin of RPI is connected to the BB using the pin in column A and row 23. As previous, two connections from that row are used for connecting the emitter to the ground. Finally, the output pin of RPi is connected to the column A in row 27. Such row will have two connections to the resistor of each circuit. Note that the connections from the RPi pins to the BB use male/female jumper wires.

The wood plank will look as given in the following figure after building the circuits, connecting them to RPi, and attaching the tires to the motors.

Note that the default state of the output pin of the GPIO is low. This means the transistor is not active. In order to activate the transistor, the state of such pin must be changed to high. To do that, we have to access the RPi in order to execute a Python script that changes the state of the GPIO output pin to high. This activates the transistors and thus runs the motors.

6. Accessing the RPi

There are different ways of accessing the RPi. One way is to connect it to a display using the HDMI port in addition to a keyboard and a mouse using 2 USB ports. Another way is just connect to using an Ethernet cable to the Ethernet interface of your PC and establish a secure session (SSH) based on the IPv4 address assigned to the RPi Ethernet interface.

For instructions about establishing an SSH session between your PC and the RPi using an Ethernet cable, read my tutorial titled “ Building an Image Classifier Running on Raspberry Pi ” available in my LinkedIn profile here: https://linkedin.com/pulse/building-image-classifier-running-raspberry-pi-ahmed-gad .

For instructions about creating a SSH session wirelessly between your PC and the RPi, read the first step “ Connecting RPi to a PC using the Wireless Interface ” of my article titled “ Building Surveillance System using USB Camera and Wireless-Connected Raspberry Pi ” which is available also in my LinkedIn profile at this link: http://www.linkedin.com/pulse/building-surveillance-system-using-usb-camera-raspberry-ahmed-gad .

After being able to establish the SSH session using the MobaXterm software, you will see the following screen assuming that the IPv4 address assigned to the RPi is 192.158.1.9.

7. Controlling Car Motors

After being able to access the RPi, we can create a Python script to control the DC motors. This is by changing the state of the GPIO output pin to high and low. You can drag and drop files from and to RPi SD card simply using the MobaXterm secure file transfer protocol (SFTP) session that opens automatically after opening the SSH session. You can also create a file directly using the “Create new file” option in that software. Whatever the way used, we need to have a Python script with the following code:

import time
import RPi.GPIO

# Initializing the GPIO pins. The numbering using is board.
RPi.GPIO.setmode(RPi.GPIO.BOARD)

# Configuring the GPIO pin number 8 to be an output pin.
RPi.GPIO.setup(8, RPi.GPIO.OUT)

print("Running Motor.")
# Running the motor by changing the GPIO output pin state to high.
RPi.GPIO.output(8, RPi.GPIO.HIGH)

# Leave the motor running for 3 seconds.
time.sleep(3)

# Stop the motor by changing the GPIO output pin state back to low.
RPi.GPIO.output(8, RPi.GPIO.LOW)

# cleanup all GPIO pins.
print("Clean Up GPIO.")
 RPi.GPIO.cleanup()

This code imports the RPi.GPIO module to access the GPIO pins. The “RPi.GPIO.setmode()” function sets the pins numbering to “RPi.GPIO.BOARD”. This means the pins numbers are assigned to the board numbering. The following figure from “ http://www.exploringrpi.com/chapter6 ” gives the numbers and locations of the GPIO pins according to board numbering. Now it is clear where exactly the pins we are using in this experiment.

The above code configures the pin number 8 as an output pin using the “RPi.GPIO.setup()” function. Then its state is changed to high using the “RPi.GPIO.output()” function. At this time, the motor will run. It will still run for three seconds according to the “time.sleep()” function. After that time, its state will be returned back to low. Finally, the pins of the GPIO are returned to their default states by cleaning their current signals using the “RPi.GPIO.cleanup()” function.

Summary of the above work can be found in this video on my YouTube channel:

8. Connecting a USB Camera to RPi

After being able to run the motors by changing the GPIO output pin state from low to high, we can extend that by using a USB camera. The camera is connected to one of the USB ports of the RPi. Based on its captured images, we can determine whether the motors will run or not.

For instructions about how to access a USB camera from the RPi, you can read the previous 2 tutorials just mentioned above. After at least reading the first tutorial titled “ Building an Image Classifier Running on Raspberry Pi ”, you should be able to capture and save images using the PyGame module and process them.

Here is the Python code used to capture and save a single image using PyGame and display it:

import os

import pygame, sys



from pygame.locals import *

import pygame.camera



width = 320

height = 240



# initialise PyGame

pygame.init()

pygame.camera.init()

cam = pygame.camera.Camera("/dev/video0",(width,height))

cam.start()



# Setup a window for displaying the captured images

windowSurfaceObj = pygame.display.set_mode((width,height),1,16)

pygame.display.set_caption('Camera')



# Take a picture

image = cam.get_image()

# Display the picture

catSurfaceObj = image

windowSurfaceObj.blit(catSurfaceObj,(0,0))

pygame.display.update()

pygame.image.save(windowSurfaceObj, './test_picture' + str(k) + '.jpg')

cam.stop()

After doing that, we are ready to continuously capture images, analyze their color, and decide whether to run or stop the motors.

9. Controlling the Motors using the Camera

Each captured image using the USB camera connected to the RPi will be processed to determine whether its dominated color is green or red. If green, then the GPIO output pin state will be made high to move the car by running its motors. If the dominating color is red, then the GPIO output pin state will be set to low to stop the car. This is similar to the traffic sign.

Here is the Python code used to determine the dominating color in the captured images and change the state of the GPIO output pin according to it:

import time

import RPi.GPIO

import numpy

import os

import pygame.camera

import pygame



#####GPIO#####

# Initializing the GPIO bins. The numbering using is board.

RPi.GPIO.setmode(RPi.GPIO.BOARD)



# Configuring the GPIO bin number 22 to be an output bin.

RPi.GPIO.setup(8, RPi.GPIO.OUT)



#####PyGame#####

# Initializing PyGame and the camera.

pygame.init()

pygame.camera.init()



# Captured image dimensions. It should be less than or equal to the maximum dimensions acceptable by the camera.

width = 320

height = 240



# Preparing a resizable window of the specified size for displaying the captured images.

window = pygame.display.set_mode((width, height), pygame.RESIZABLE)



# Specifying the camera source and the image dimensions.

cam = pygame.camera.Camera("/dev/video0", (width, height))

cam.start()



for im_num in range(0, 2000):

print("Image : ", im_num)



    im = cam.get_image()



# Displaying the image on the window starting from the top-left corner.

window.blit(im, (0, 0))



# Refreshing the window.

pygame.display.update()



    im = pygame.surfarray.array3d(window)

    r = numpy.mean(im[:, :, 0])

    g = numpy.mean(im[:, :, 1])

    b = numpy.mean(im[:, :, 2])



if ((r - g) > 30 and (r - b) > 30):

print("Red - Stop")

try:

            RPi.GPIO.output(8, RPi.GPIO.LOW)

except KeyboardInterrupt:  # CTRL+C

print("Keyboard Interrupt.")

except:

print("Error occurred.")

elif ((g - r) > 30 and (g - b) > 30):

print("Green - Move")

try:

            RPi.GPIO.output(8, RPi.GPIO.HIGH)

except KeyboardInterrupt:  # CTRL+C

print("Keyboard Interrupt.")

except:

print("Error occurred.")



# Stopping the camera.

cam.stop()



# cleanup all GPIO bins.

print("Clean Up GPIO.")

RPi.GPIO.cleanup()

Simply, the average of each of the three RGB channels is calculated. If the average value of the red channel is larger than the average color of both the blue and green channels, then the dominating color is red and the car will stop. If the average color of the green channel is larger than the average of the other 2 channels, then the car moves. Note that the if statements use a margin of 30. This makes the decision more accurate.

After running this code, a window will appear that shows each captured image. When the dominant color in the image is green, the motors run and a print message appears on the terminal of the session. This is also the same for red.

I prepared two images, one red and other is green as given in the following figure. That image is displayed on the screen of the PC. When the camera is placed in front of the green one, the motors run. Motors stop when in front of the red one.

The following video shows that the experiment works well and the color is classified correctly.

For More Details

“ Building an Image Classifier Running on Raspberry Pi ” https://linkedin.com/pulse/building-image-classifier-running-raspberry-pi-ahmed-gad .
“ Building Surveillance System using USB Camera and Wireless-Connected Raspberry Pi ” http://www.linkedin.com/pulse/building-surveillance-system-using-usb-camera-raspberry-ahmed-gad

For Contacting the Author

E-mail: [email protected]
LinkedIn: https://linkedin.com/in/ahmedfgad/
KDnuggets: https://kdnuggets.com/author/ahmed-gad
YouTube: http://youtube.com/AhmedGadFCIT
TowardsDataScience: https://towardsdatascience.com/@ahmedfgad

Building Vision-Controlled Car using Raspberry Pi