FaaSMark is a benchmark for serverless compute platforms. It is designed to measure function invocation latency across different clouds and using different invocation methods and function parameters.

The benchmark currently tests latency under different:

• Invocation mathods (HTTP vs SDK)
• Programming languages
• Memory sizes
• Load conditions
• FaaS providers

This version supports aspect (invocation memory, programming languages and memory size) benchmarks and concurrency benchmarks only on AWS Lambda.

Benchmark code is divided into three parts:

0. Client is the code that drives the benchmark process. It can run on your local machine or any cloud VM/container.
1. Initiator is a FaaS function that is deployed on the platform being benchmarked. The initiator is responsible for repeatedly invoking empty functions (see below) and measuring their invocation latencies. There are two types of initiators: simpleInitiator and concurrencyInitiator. The first is used for measuring different aspects of function invocation (method, languages and memory sizes) and comparing different cloud platforms. The second is used for load tests.
2. Empty is a FaaS function that retuns immediately after invocation. This function is deployed on the FaaS platform being benchmarked. Empty is implemented is JavaScript across on platforms and in multiple languages on AWS Lamda. Concurrency tests use a more complex version of empty called sleeper that can sleep for a specified interval before retuning. This feature is used for warming up multiple functions containers before starting the benchmark itself.

Deploying both initiator and empty on the same FaaS platforms assures that no factors outside of the platform affect invocation latency.

In order to perform the benchmark you first need to deploy functions to the different FaaS provider platforms. See how to below.

Invoke the benchmark using the command

node faasmark.js

Benchmark behavior is control by the file settings.json which has the following fields:

Note that automatic deployment onto Azure and Bluemix is not supported yet, and that their settings are currently hard coded.

Deployment method varies between clouds.

Use the Serverless Framework (sls) to deploy functions to AWS Lambda. Set your AWS credentials and install install sls. Now run:

cd providers/aws
sls deploy

Automatic deployment is not yet supported on Azure. To deploy on Azure Functions you can use the web console to create two functions with the contents of

providers/azure/empty.js
providers/azure/simpleInitiator.js

Use the file name (minus the extension) as the function name and configure HTTP triggers for the functions.

See Azure above.

Once the gcloud utility is installed and configured, you can use the

providers/gcloud.js

utility to create a project, configure it to run functions and deploy functions to that project.

You can use an existing project or create a new one. Once you have a project name you must set the GCLOUD_REGION and GCLOUD_PROJECT variables accordingly.