1. Redis: Install Redis on your machine following instructions from here

2. Python3.6

3. Install python modules:

   pip install -r requirements.txt
   

To run the pipeline:

python3 pipeline.py

The default behavior is for the pipeline to kick off a 4 chains of tasks, launch all tasks dispatched to it via the Redis queues and stop listening on the queue. This allows the pipeline to not block indefinitely on the redis queue.

To block on the redis_q use the -b flag to the pipeline:

python3 pipeline.py -b

Once the pipeline is running in blocking mode, you may send as many params as you wish via the pump.py utility. Separate from the pipeline running, you may pipe a simulated parameter into the pipeline from the commandline utility

python3 pump.py -p 100

To stop a pipeline that is running in blocking mode run

python3 pump.py -p None