Giter Club home page Giter Club logo

nativescript-tfjs-sample's Introduction

NativeScript TensorflowJS Demo

Documentation - WIP

How to run this project

  • Install NativeScript 5.x
  • Clone this repo
  • Create a firebase app, and download your google-services.json
  • npm i
  • run this via
    • npm run android
    • npm run ios

That's about it, you're good to go. Explore the NativeScript + TFjs examples.

How to install TFjs into your NativeScript Application

You have 2 methods to do this each with its own pros and cons,

Method 1: Install as an npm package

To install TFjs into your NativeScript app, just run npm i -S @tensorflow/tfjs.

After installing this, your app builds will work fine, but if you try to import anything from @tensorflow/tfjs, your app crashes, because Tensorflow JS does not understand NativeScript environment. To overcome this, we're just going to tell Tensorflow JS that it is running inside a Node.js environment. To do this,

In your app.ts (or main.ts or app.vue) add the following lines,

//noinspection JSUnresolvedVariable,JSUnusedLocalSymbols
global.setImmediate = setTimeout;
global.clearImmediate = clearTimeout;
global.performance = {
  now() {
    if (global.android) {
      return java.lang.System.nanoTime() / 1000000;
    } else {
      return CACurrentMediaTime();
    }
  }
};
global.process = {
  versions: {
    node: '10.0'
  }
};

Tensorflow also requires setImmediate and clearImmediate methods, in case of NativeScript, just aliasing them to setTimeout and clearTimeout does the job. It also requires a way to get current system time via performance.now() method, in case of NativeScript we're just tapping into Android/iOS Native APIs to fetch the same. After that you can see that we're creating a new global variable called process, if you're already familiar with Node.js you might know that process is the global object in Node. All that Tensorflow checks to figure out the environment it is in is to check process.versions.node, so we've defined just that and nothing else.

With that in place, when you import something from @tensorflow/tfjs your app no longer crashes.

Pros of Method 1

  • Easiest way to get started with using TFjs in NativeScript
  • Faster training performance
  • No WebView needed
  • You're directly using Tensorflow JS with JavaScript runtime of the device
  • Good for small datasets, where you can afford to train everytime the app opens, because it usually takes only 2 to 3 seconds for small datasets (upto a few thousand data points)

Cons of Method 1

  • No support to save/load a trained model, because the save/load implementations in TFjs are currently relying on browser APIs, which aren't available in NativeScript.
  • You can't do image, audio or video processing because, the APIs of TFjs that do these kind of tasks expect to receive a HTML Image, Audio or Video element, which isn't available when working with Pure NativeScript.

Method 2: Use it inside a WebView and communicate to it via {N} WebView interface

To use TFjs from a WebView in your NativeScript app, create a www (or any name), directory inside your app folder, this is where the webpages where TFjs is used will live. Inside www folder, create a lib folder, and place tf.min.js inside it, so that the machine learning capabilities can be used offline, and there is no download time when the WebView loads. If you are using any other supporting libraries for Tfjs for example, Mobilenet model, Knn-classifier, etc, you can place them in this lib folder. Next, create HTML files inside www folder which import these libraries.

Next, setup the NativeScript WebView interface,

tns plugin add nativescript-webview-interface
cp node_modules/nativescript-webview-interface/www/nativescript-webview-interface.js app/www/lib/

# replace the destination folder names with the one relavant to your project

Refer the README of nativescript-webview-interface to understand how to community between NativeScript and a WebView.

Pros of Method 2

  • Let's you utilize everything that TFjs has to offer

Cons of Method 2

  • Can be a bit intimidating to setup the first time.

Is it worth training models on device

WIP

How to use existing model to perform inference

WIP

How to use ML Kit

Refer the README of nativescript-plugin-firebase

Final Thoughts

WIP

nativescript-tfjs-sample's People

Contributors

shiv19 avatar billn6 avatar

Stargazers

Nicholas Gunner avatar Alexey Okhrimenko avatar Jen Looper avatar Petr Mitev avatar  avatar shilik avatar

Watchers

James Cloos avatar shilik avatar  avatar  avatar

nativescript-tfjs-sample's Issues

Gradle build error

The Android Gradle plugin supports only Crashlytics Gradle plugin version 1.25.4 and higher. Project 'tFlow' is using version 1.25.1.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.