Posted by Stan Carney on Jan 17, 2013 in Programming | 0 comments

Django has several asynchronous processing options available like Celery, Chronograph and Django Tasks. They all offer lots of features but if you want something simpler, maybe just the ability to send a webhook on the submission of a form without having the user wait around for the webhook to complete (or the remote site timeout), it seems like overkill.

As a result I came up with the following simple Django asynchronous processing application. It was written to work with Django 1.3 but works with Django 1.4’s, sans the new select_for_update functionality.

There is just one file called async_runner.py. It’s a Django command that queries the database frequently (every 1/2 a second + execution time) looking for new unexecuted Tasks, it locks the row at the database level, executes the embedded Python script via exec(task.script) and marks the row as executed before continuing. If the Task crashes it attempts to mark it as executed so it won’t crash again. If 3 consecutive tasks crash, the time in which the loop sleeps increases.

One or many async_runners can be started via ./manage.py async_runner start via the console or init scripts.

Code wishing to execute things asynchronously would create a new Task object and save it:

Task.objects.create(script=”from myapp import api; api.send_webhook_notifications([%s])” % user.id)

It isn’t really suited for the heavy lifting or complicated processing that something like Celery is capable of but for simple things, like sending a webhook in which the initiating code just calls and forgets, it works quite well.