Giter Club home page Giter Club logo

react-tensorflow-object-detection's Introduction

caMicroscope GSoC 2020 Code Challenge:

Using a machine learning toolkit of your choice, create a tool which identifies objects in the image, then returns positions in pixels corresponding to bounding boxes of a user-selected class of object in the image. For example, given an image with both cats and dogs, return bounding boxes for only cats.

working_gif

Added a Specific Class Input to enter one of the 90 classes in the COCO dataset. The predictions will be filtered and only that class will be shown.

model
  .detect(document.getElementById("webcamFeed"))
  .then(predictions => {
    if (specificClass === "") {
      setPredictions(predictions);
    } else {
      const specificDetections = predictions.filter(
        p => p.class === specificClass
        );
      console.log(specificDetections);
      setPredictions(specificDetections);
    }
  })

Live Demo - Click Here

  1. Wait for the Model to load
  2. Allow access to the Webcam
  3. Click the 'Capture Photo' button to log a base64 string of the Webcam frame
  4. Click 'Predict' button to get a Predictions
  5. Type a Class Name to filter the Predictions for that specific Class

Note: Please Click the canvas to load a Webcam frame onto the canvas. Sorry but I couldn't make the useRef() hook work correctly. I will fix this soon!


React.js and Tensorflow.js Object Detection

A React Web Application to test the COCO SSD pre-trained model to detect objects through the User's Webcam or other video source. Uses Tensorflow.js to run the model directly in the browser.

working

Object detection model that aims to localize and identify multiple objects in a single image.

This model is a TensorFlow.js port of the COCO-SSD model. For more information about Tensorflow object detection API, check out this readme in tensorflow/object_detection.

This model detects objects defined in the COCO dataset, which is a large-scale object detection, segmentation, and captioning dataset. You can find more information here. The model is capable of detecting 90 classes of objects. (SSD stands for Single Shot MultiBox Detection).

Usage

const [model, setModel] = useState(null);
const [predictions, setPredictions] = useState(null);

useEffect(() => {
    const loadModel = async () => {
      const newModel = await cocoSsd.load({ base: "lite_mobilenet_v2" });
      setModel(newModel);
    };
    loadModel().then(setIsModelLoaded(true));
  }, []);

model.detect(document.getElementById("webcamFeed"))
    .then(predictions => {
        setPredictions(predictions);
    })

Sample Output of the model.detect() method

[{
  bbox: [x, y, width, height],
  class: "person",
  score: 0.8380282521247864
}, {
  bbox: [x, y, width, height],
  class: "kite",
  score: 0.74644153267145157
}]

Future Improvements:

  1. Because of Tensorflow.js we can offer a option to the user to dynamically change the model which is being used for Object Detection.
  2. The user can save the image with detections with a Right Click > Save Image As. Adding a dedicated Download button will simplify this process.
  3. As we have the box co-ordinates of the detected object which we use to paint in the canvas, we should be able to cut out the portion of the image and offer cropped views of the image for the user.

Resources:

References:

Available Scripts:

In the project directory, you can run: yarn start

Open http://localhost:3000 to view it in the browser.


This project was bootstrapped with Create React App.

react-tensorflow-object-detection's People

Contributors

amogh-w avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.