Giter Club home page Giter Club logo

alligator's Introduction

Alligator

https://travis-ci.org/toastdriven/alligator.png?branch=master

Simple offline task queues. For Python.

"See you later, alligator."

Latest documentation at http://alligator.readthedocs.org/en/latest/.

Requirements

  • Python 2.6+ or Python 3.3+
  • (Optional) redis for the Redis backend
  • (Optional) beanstalkc for the Beanstalk backend
  • (Optional) PyYAML for the Beanstalk backend

WHY?!!1!

  • Because I have NIH-syndrome.
  • Or because I longed for something simple (~375 loc).
  • Or because I wanted something with tests (90%+ coverage) & docs.
  • Or because I wanted pluggable backends.
  • Or because testing some other queuing system was a pain.
  • Or because I'm an idiot.

Basic Usage

This example uses Django, but there's nothing Django-specific about Alligator.

I repeat, You can use it with any Python code that would benefit from background processing.

from alligator import Gator

from django.contrib.auth.models import User
from django.shortcuts import send_email


# Make a Gator instance.
# Under most circumstances, you would configure this in one place &
# import that instance instead.
gator = Gator('redis://localhost:6379/0')


# The task itself.
# Nothing special, just a plain *undecorated* function.
def follow_email(followee_username, follower_username):
    followee = User.objects.get(username=followee_username)
    follower = User.objects.get(username=follower_username)

    subject = 'You got followed!'
    message = 'Hey {}, you just got followed by {}! Whoohoo!'.format(
        followee.username,
        follower.username
    )
    send_email(subject, message, '[email protected]', [followee.email])


# An simple, previously expensive view.
@login_required
def follow(request, username):
    # You'd import the task function above.
    if request.method == 'POST':
        # Schedule the task.
        # Use args & kwargs as normal.
        gator.task(follow_email, request.user.username, username)
        return redirect('...')

Running Tasks

Rather than trying to do autodiscovery, fanout, etc., you control how your workers are configured & what they consumer.

If your needs are simple, run the included latergator.py worker:

$ python latergator.py redis://localhost:6379/0

If you have more complex needs, you can create a new executable file (bin script, management command, whatever) & drop in the following code.

from alligator import Gator, Worker

# Bonus points if you import that one pre-configured ``Gator`` instead.
gator = Gator('redis://localhost:6379/0')

# Consume & handle all tasks.
worker = Worker(gator)
worker.run_forever()

License

New BSD

Future Wishlist

These things aren't present yet, but maybe someday they will be.

# Delayed tasks (run in an hour).
with gator.options(run_after=60 * 60) as task:
    task(this_can_wait)

# Dependent tasks, will only run if the listed tasks succeed.
# Maybe.
with gator.options(depends_on=[feeds_job]) as task:
    task(rebuild_cache)

Running Tests

Alligator has 95%+ test coverage & aims to be passing/stable at all times.

If you'd like to run the tests, clone the repo, then run:

$ virtualenv env2
$ . env2/bin/activate
$ pip install -r requirements.txt
$ python setup.py develop
$ py.test -s -v --cov=alligator --cov-report=html tests

TODO

  • Scheduled tasks
  • Dependent tasks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.