sjakthol / aws-compute-benchmarks Goto Github PK

Benchmarks for Amazon EC2, AWS Fargate and AWS Lambda.

• Benchmark single-CPU performance of Amazon EC2, AWS Fargate and AWS Lambda environments.

Here are some results from a benchmark executed in eu-west-1 region on 2019-12-22.

Footnotes:

1. Operations per second. Higher is better (see below for methodology)
2. Fargate Runtime with 1 vCPU & 2,048 MB of memory
3. Lambda Runtime with 1 vCPU equivalent of compute power (1,972 MB)

You must have the following setup to be able to run these benchmarks:

• AWS credentials with admin level privileges
• AWS account with VPC and Subnets from sjakthol/aws-account-infra.
• Docker

Once setup, you can deploy the benchmark infra (ECR registry) by running

make deploy-infra

in this directory. Once the ECR registry has been created, you'll need to build the Docker image used for benchmarks:

make login build tag push

You can execute benchmarks as follows:

• AWS Lambda Benchmark:
  • Deploy: make deploy-lambda
  • Execute: make lambda-invoke
• AWS Fargate Benchmark
  • Deploy: make deploy-fargate
  • Execute: Executed automatically as ECS Service. Executes continuously until deleted (or manually scaled to 0).
• Amazon EC2
  • Deploy: make deploy-ec2
  • Execute: Executed automatically. Instance terminated after execution.

The benchmark results can be found from Amazon CloudWatch Log Groups with similar names.

Execute the following commands to clean up the relevant resources (also destroys log groups with results):

make delete-lambda
make delete-fargate
make delete-ec2
make delete-infra # NOTE: ECR registry must be cleaned manually before deleting the infra.

The raw, single thread CPU performance is measured with a Node.js application. The application computes PBKDF2 hashes in a loop and measures the number of hashes computed per second (the higher the better). Each run lasts for 15 seconds. The code is available in benchmark/ directory.