Giter Club home page Giter Club logo

pandas-sample-2's Introduction

Installation & usage instructions

Development

Install pandas locally

sudo python3 -m pip install --upgrade pandas

Create a python3 virtualenv locally*

virtualenv -p python3 .

Activate virtualenv

source bin/activate

or (based upon which shell you're using)

source bin/activate.fish

pip3 install pandas

* installing the virtualenv tool is different per operating system, google how to do this for yours

Deployment

Create package (run from app/)

zip -g PandasSample.zip main.py

Add libraries (run from lib/python3.6/site-packages**)

zip -r9 ../../../app/PandasSample.zip .

** this directory may be different for various operating systems, see here

Below is deprecated. This is the final command to create a zip in one line, when running Linux with the Fish shell interpreter, from app/. zip -g PandasSample.zip main.py; and zip -r9 PandasSample.zip ../lib/python3.6/site-packages/

Notes

There are 4 different testing scenario's to accomplish the use case @ Peter's mail:

    1. Testing the lambda locally using a locally-defined test event (in the python script). This is the first stage. Everything should work here before moving on. We're verifying if the base functionality of the script works here.
    1. Testing the lambda directly from the web interface using a there-defined test event. This is the second step. Upload the local environment to aws lambda, check if it works there. It's required to replicate the test event locally into a json test event on the web interface here.
    1. Testing the lambda via the web interface using an API gateway. This adds another conversion layer to the test event. This means that the initial test event is not as you define it. When you add a way to connect to the lambda using the HTTP protocol, so by using the AWS API gateway, this gateway transforms the event you sent in the http post body. It adds a lot of meta data to it, like this:
            "resource": "/myPandasFunction",
            "path": "/myPandasFunction",
            "httpMethod": "POST",
            "body": "{\n        \"seriesStringsTitle\": \"A Series Of Strings\",\n        \"seriesNumbersTitle\": \"A Series Of Numbers\",\n        \"seriesStrings\": [\"value1\", \"value2\", \"value3\", \"value4\"],\n        \"seriesNumbers\": [1, 2, 3, 4, 5]\n    }",
            "isBase64Encoded": False
    }

Where body in input is the only thing you've passed it (through the http post request body). It's necessary to change the python script after adding an api gateway, because the event param now contains all of the extra meta data, instead of just the previous event. (It's also necessary to change the test event when directly testing the lambda using the lambda test web interface, because the script will now expect the meta data.)

    1. Testing the lambda via CURL using an API gateway. For some reason there's a difference between testing the api gateway through curl (or any other http client) and the amazon api gateway testing interface.

pandas-sample-2's People

Contributors

edwinvanrooij avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.