Giter Club home page Giter Club logo

serverless-multilabel-text-classification's Introduction

Magpie serverless

Trying to deploy in a serverless fashion this framework. The serverless function works locally but still needs some tweaks to work in the cloud. The model folder contains the pretrained model for the function to work. Pull request highly appreciated.

Status

The package is to heavy for aws lambda even with some tweaks by reducing dependencies size (get rid of shared libraries etc). I've not tested on kubeless. I keep the repository up for the records

Prerequesites

  • Have node and npm installed. There is a good guide for installing node and npm on linux here
  • Have python and pip installed. There is a good guide for installing python with conda here
  • Have virtualenv installed:
 pip install virtualenv
  • Make sure you have exported your AWS keys in your environmnet variables

Install / Update serverless framework

npm install -g serverless

Getting started

git clone https://github.com/Sach97/serverless-multilabel-text-classification.git
cd serverless-multilabel-text-classification
virtualenv venv --python=python3
source venv/bin/activate
pip3 install git+https://github.com/inspirehep/[email protected] && pip3 install tensorflow
serverless invoke local -f predict --data '{"text":"Stephen Hawking studies black holes"}' --log # uncomment the two lines in handler.py

Build the code dependcy package yourself + train model

git clone https://github.com/Sach97/serverless-multilabel-text-classification.git
cd serverless-multilabel-text-classification
chmod +x build_vendored.sh
chmod +x clean_venv.sh
chmod +x env_var.sh
virtualenv venv --python=python3
source venv/bin/activate
pip3 install git+https://github.com/inspirehep/[email protected] && pip3 install tensorflow
sh clean_venv.sh
sh build_vendored.sh
python train_model.py
python upload_model.py
serverless invoke local -f predict --data '{"text":"Stephen Hawking studies black holes"}' --log 

Run locally

serverless invoke local -f predict --data '{"text":"Stephen Hawking studies black holes"}' --log

TODOs

  • Create a real function not just an import magpie
  • Make a better shell script for the zip
  • Resolve the issue
  • Add CircleCI continuous integration badge and an explanation guide.
  • Add an AWS deployment button
  • Load the model globally before a lambda function get called, like in this repo

serverless-multilabel-text-classification's People

Contributors

sachaarbonel avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

serverless-multilabel-text-classification's Issues

Unable to import module 'handler': No module named 'magpie'

After deploy when you try to invoke the lambda function like this:

serverless invoke -f predict --data '{"text":"Stephen Hawking studies black holes"}' --log

You'll get this error:

Unable to import module 'handler': No module named 'magpie'

A possible solution add:

from __future__ import absolute_import

on top of init.py of the magpie package

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.