Query the XKCD API.
This is a learning project. The XKCD web comic provides a JSON API for requesting comic strips. This app, while being intentionally over-engineered, demonstrates some technologies I want to understand better, and serves as a showcase/reference for tools like FastAPI, rabbitMQ, minio, Redis, and more.
The main app consists of:
- An API (FastAPI)
- A SQLite database for storing comics, images, and metadata
- A rabbitMQ messaging service
- A Redis backend for rabbitMQ job results
- I am sticking with Redis for now, despite their recent license change. I have not tested running Celery with a different key/value store.
- I would like to eventually switch to Valkey
- A Celery app to handle background/async tasks with rabbitMQ and Redis
The API handles requests to the XKCD API, with a rate limit of 1 request per 5 seconds (to avoid sending a ton of requests to the wonderful Randall Monroe...he doesn't deserve that) when multiple comics are requested. In the case of multiple comic requests, requests are broken into "partitions" when >15 comics are requested at once, and the requests are run as background tasks with rabbitMQ; the results can be retrieved with the FastAPI app, which pulls the results from the Redis backend.
-
Edit environment variables in
./containers/env_files
-
From the project root, run
python dockerctl.py
- Because the
docker-compose.yml
and.env
files are nested in./containers
, it's annoying having to prepend-f ./containers/docker-compose.[dev|prod].yml
to ever command. - The
dockerctl.py
script uses the cross-platformsubprocess
module to run Docker commands on your behalf. Follow the prompts to control Docker operations
- Because the
-
Visit the API, rabbitMQ management console, or Redis commander using the following (assumes default ports, substitute your own if you change them):
service | port |
---|---|
FastAPI | 8000 |
rabbitMQ Management Console | 15672 |
redis-commander webUI | 8081 |
This app uses Celery for background & scheduled tasks. Because the code is in a src/
directory, the celery -A ...
command needs to be modified. To change the working directory of celery
, use the --workdir=src/app_name
param:
celery --workdir=src/auto_xkcd -A ...