You will need to have Docker compose installed

Setup the .env file:

1. cp .env.example .env
2. Fill in your AWS S3_KEY and S3_SECRET values
3. Create a new SECRET_KEY - this can be any random string

Bring all of the containers up (this runs in the foreground until you hit ctrl-c):

npm run up

With npm run up still going, setup the development and test databases

npm run setup

npm run ssh

npm test

While testing you may mess up the test database, and start getting database errors while running your tests.

Then you need to run npm run setup to reset the test database.

npm run lint

npm run migration -- -m create_logs

npm run migrate

npm run rollback

npm run db

Supply a name for the client like dumbledore:

npm run api-token -- mylittledrone

The app requires auth tokens in the header with every API request. All requests are logged.

We could scale horizontally with this system. If database writes became a bottleneck, we could stagger log ingestions or shard the database. Using AWS lambda would also be advisable for log ingestion.

We would implement hashes of all logs to verify proper upload from drones.

Furthermore, every log would be saved in its original format in s3, replicated across a geographically distinct zone. We could always compare hashes of the logs in the db with the files to ensure consistency. And if the db ever disappeared, we could reload the log events from s3.

We'd aim for unit and request tests to ensure proper API responses. To test capacity of the system, we'd simulate mass uploads with apache benchmark or similiar scripts.