Home

Pyres is a Resque clone built in python. Each job in the queue corresponds to a python class that needs to define a perform method.

Let’s say I have a blog which needs to check comments for spam. When the comment is saved in the DB we create a job in the queue with comment data. Looks something like.

classComment(Model):
name=Model.CharField()
email=Model.EmailField()
body=Model.TextField()
spam=Model.BooleanFieldqueue="Spam"#@staticmethoddefperform(comment_id):
//YouORMorSQLtogetthecommentdatafromthecomment_idx=urllib.open("http://apikey.rest.akismet.com/1.1/comment-check?comment_author="+self.name.. soforthifx=="true":
//savethecommentspamfieldtotrue.
else:
//commentisfine.

You can convert your existing class to be compatible with Pyres. All you need to do is add a queue variable and define a perform method on the class.

To insert a job into the queue you need to do something like this:

frompyresimportResQr=Resq()
r.enqueue(Comment, 23)

This puts a job into the queue Spam. Now we need to fire off our workers. In the scripts folder there is an executable pyres_worker which is used to start a worker.

$ ./pyres_worker Spam

Just pass a comma seperated list of queues the worker should poll.

Tips and Tricks

Developing Locally

While developing locally, it’s uncommon to need to rely on a separate worker process that watches the queues. Instead, it’s better to run the job synchronously. You can do this with a simple superclass that your jobs will inherit from:

frompyresimportResQr=Resq()
classJobBase(object):
@classmethoddefenqueue(cls, *args):
ifapp.debug:
cls.perform(*args)
else:
r.enqueue(cls, *args)
classGenerateThumbnailsJob(JobBase):
pass

From then on, you can enqueue jobs one way regardless if you’re in development, testing or production mode:

fromjobsimportGenerateThumbnailsJobGenerateThumbnailsJob.enqueue(*args)