[Python]Live object detection with webcam using Tensorflow API

Hello everyone, my name is Nitro and welcome to Tensorflow object detection tutorial.

In this tutorial, we are going to make live object detection through webcam with Tensorflow API.

And it is my first time to translate my post into English, so I hope you guys can understand and follow well.

If you want to see Korean post, then please visit here.

Actually, all of the source code is already on GitHub, so just download it and edit some of the code.

1. Download Code

https://github.com/tensorflow/models

Visit above URL link and download it on your desktop.

2. Unzip and copy

After finished download, then you have a folder named "object_detection" in "models/research/" path.

Just copy that folder somewhere you can easily access.

3. Edit source code

Now, it is time to rewrite source code.

This example is consist of notebook file, so we should convert to .py file. 

But... just make it.

source code

import numpy as np
import os
import six.moves.urllib as urllib
import sys
import tarfile
import tensorflow as tf
import zipfile

from collections import defaultdict
from io import StringIO
from matplotlib import pyplot as plt
from PIL import Image

import cv2
#VideoCapture(0) means index of webcam, it increase by recognized other webcam.
cap = cv2.VideoCapture(0)

sys.path.append("..")

from utils import label_map_util

from utils import visualization_utils as vis_util

MODEL_NAME = 'ssd_mobilenet_v1_coco_11_06_2017'
MODEL_FILE = MODEL_NAME + '.tar.gz'
DOWNLOAD_BASE = 'http://download.tensorflow.org/models/object_detection/'

PATH_TO_CKPT = MODEL_NAME + '/frozen_inference_graph.pb'

PATH_TO_LABELS = os.path.join('data', 'mscoco_label_map.pbtxt')

NUM_CLASSES = 90

opener = urllib.request.URLopener()
opener.retrieve(DOWNLOAD_BASE + MODEL_FILE, MODEL_FILE)
tar_file = tarfile.open(MODEL_FILE)
for file in tar_file.getmembers():
  file_name = os.path.basename(file.name)
  if 'frozen_inference_graph.pb' in file_name:
    tar_file.extract(file, os.getcwd())

detection_graph = tf.Graph()
with detection_graph.as_default():
  od_graph_def = tf.GraphDef()
  with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
    serialized_graph = fid.read()
    od_graph_def.ParseFromString(serialized_graph)
    tf.import_graph_def(od_graph_def, name='')

label_map = label_map_util.load_labelmap(PATH_TO_LABELS)
categories = label_map_util.convert_label_map_to_categories(label_map, max_num_classes=NUM_CLASSES, use_display_name=True)
category_index = label_map_util.create_category_index(categories)

def load_image_into_numpy_array(image):
  (im_width, im_height) = image.size
  return np.array(image.getdata()).reshape(
      (im_height, im_width, 3)).astype(np.uint8)

PATH_TO_TEST_IMAGES_DIR = 'test_images'
TEST_IMAGE_PATHS = [ os.path.join(PATH_TO_TEST_IMAGES_DIR, 'image{}.jpg'.format(i)) for i in range(1, 3) ]

IMAGE_SIZE = (12, 8)

with detection_graph.as_default():
  with tf.Session(graph=detection_graph) as sess:
    while True:
      ret, image_np = cap.read()
      image_np_expanded = np.expand_dims(image_np, axis=0)
      image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')
      boxes = detection_graph.get_tensor_by_name('detection_boxes:0')
      scores = detection_graph.get_tensor_by_name('detection_scores:0')
      classes = detection_graph.get_tensor_by_name('detection_classes:0')
      num_detections = detection_graph.get_tensor_by_name('num_detections:0')
      (boxes, scores, classes, num_detections) = sess.run(
          [boxes, scores, classes, num_detections],
          feed_dict={image_tensor: image_np_expanded})
      vis_util.visualize_boxes_and_labels_on_image_array(
          image_np,
          np.squeeze(boxes),
          np.squeeze(classes).astype(np.int32),
          np.squeeze(scores),
          category_index,
          use_normalized_coordinates=True,
          line_thickness=8)

      cv2.imshow('object detection', cv2.resize(image_np, (800,600)))
      if cv2.waitKey(25) & 0xFF == ord('q'):
        cv2.destroyAllWindows()
        break

#VideoCapture(0) is means index of the webcam, it increases by recognized other webcam.

Focus on the above sentence, VideoCapture(Number) is declaration webcam, and Number means the index of the webcam.

If you have two or more webcam, maybe you need to change the Number

And save this file in "object_detection" folder.

It doesn't matter what name is, I just named it "Object.py"

4. Install module

There’s something else at play here.

Of course, we need to download imported modules too.

In Python we can use pip, so let's use it!

numpy
six
tensorflow
matplotlib
image
opencv-python
finished

This list shows us the modules what we need, and I made the bath file to download automatically.

install help.bat

Use this!

5. Run

The source code, we made it before, move to "object_detection" and run!

(This photo is mosaicized to protect privacy.)

It takes a while and then comes out on the screen like this!

I want to make a driverless car system only use webcam and computer... but it is too hard to make.

maybe need some lidar...

Anyway, I hope you guys enjoy.

Translation

Korean Ver

[Programing/Python] - [Python]웹캠을 이용한 실시간 사물 감지 프로그램 만들기 (텐서플로우 API)