fork/django-q2
1
0
Fork 0
mirror of https://github.com/django-q2/django-q2.git synced 2026-09-29 05:08:13 +08:00
Code Issues Packages Projects Releases Wiki Activity

Replaces async occurrences with alternatives

Browse Source 
* async is now a reserved word in python3.7
 * Rename async function to enqueue
 * Rename all async_ functions to enqueue_
 * Rename Async class to AsyncTask
 * Updates the docs.
This commit is contained in:
Pierre-Elliott Bécue
2018-07-07 21:33:43 +02:00
parent 2040771e80
commit ae75a84f7f
16 changed files with 190 additions and 190 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/examples.rst
+44 -44
View File
@@ -12,18 +12,18 @@ Sending an email can take a while so why not queue it:
# Welcome mail with follow up example
from datetime import timedelta
from django.utils import timezone
from django_q.tasks import async, schedule
from django_q.tasks import enqueue, schedule
from django_q.models import Schedule
def welcome_mail(user):
msg = 'Welcome to our website'
# send this message right away
async('django.core.mail.send_mail',
'Welcome',
msg,
'from@example.com',
[user.email])
enqueue('django.core.mail.send_mail',
'Welcome',
msg,
'from@example.com',
[user.email])
# and this follow up email in one hour
msg = 'Here are some tips to get you started...'
schedule('django.core.mail.send_mail',
@@ -51,7 +51,7 @@ A good place to use async tasks are Django's model signals. You don't want to de
from django.contrib.auth.models import User
from django.db.models.signals import pre_save
from django.dispatch import receiver
from django_q.tasks import async
from django_q.tasks import enqueue
# set up the pre_save signal for our user
@receiver(pre_save, sender=User)
@@ -64,7 +64,7 @@ A good place to use async tasks are Django's model signals. You don't want to de
# has his email changed?
if not user.email == instance.email:
# tell everyone
async('tasks.inform_everyone', instance)
enqueue('tasks.inform_everyone', instance)
The task will send a message to everyone else informing them that the users email address has changed. Note that this adds almost no overhead to the save action:
@@ -87,8 +87,8 @@ The task will send a message to everyone else informing them that the users emai
for u in User.objects.exclude(pk=user.pk):
msg = 'Dear {}, {} has a new email address: {}'
msg = msg.format(u.username, user.username, user.email)
async('django.core.mail.send_mail',
'New email', msg, 'from@example.com', [u.email])
enqueue('django.core.mail.send_mail',
'New email', msg, 'from@example.com', [u.email])
Of course you can do other things beside sending emails. These are just generic examples. You can use signals with async to update fields in other objects too.
@@ -104,19 +104,19 @@ In this example the user requests a report and we let the cluster do the generat
.. code-block:: python
# Report generation with hook example
from django_q.tasks import async
from django_q.tasks import enqueue
# views.py
# user requests a report.
def create_report(request):
async('tasks.create_html_report',
request.user,
hook='tasks.email_report')
enqueue('tasks.create_html_report',
request.user,
hook='tasks.email_report')
.. code-block:: python
# tasks.py
from django_q.tasks import async
from django_q.tasks import enqueue
# report generator
def create_html_report(user):
@@ -127,16 +127,16 @@ In this example the user requests a report and we let the cluster do the generat
def email_report(task):
if task.success:
# Email the report
async('django.core.mail.send_mail',
'The report you requested',
task.result,
'from@example.com',
task.args[0].email)
enqueue('django.core.mail.send_mail',
'The report you requested',
task.result,
'from@example.com',
task.args[0].email)
else:
# Tell the admins something went wrong
async('django.core.mail.mail_admins',
'Report generation failed',
task.result)
enqueue('django.core.mail.mail_admins',
'Report generation failed',
task.result)
The hook is practical here, because it allows us to detach the sending task from the report generation function and to report on possible failures.
@@ -152,12 +152,12 @@ here's an example of how you can have Django Q take care of your indexes in real
from .models import Document
from django.db.models.signals import post_save
from django.dispatch import receiver
from django_q.tasks import async
from django_q.tasks import enqueue
# hook up the post save handler
@receiver(post_save, sender=Document)
def document_changed(sender, instance, **kwargs):
async('tasks.index_object', sender, instance, save=False)
enqueue('tasks.index_object', sender, instance, save=False)
# turn off result saving to not flood your database
.. code-block:: python
@@ -177,7 +177,7 @@ here's an example of how you can have Django Q take care of your indexes in real
index.update_object(instance, using=backend)
Now every time a Document is saved, your indexes will be updated without causing a delay in your save action.
You could expand this to dealing with deletes, by adding a ``post_delete`` signal and calling ``index.remove_object`` in the async function.
You could expand this to dealing with deletes, by adding a ``post_delete`` signal and calling ``index.remove_object`` in the enqueue function.
.. _shell:
@@ -187,13 +187,13 @@ You can execute or schedule shell commands using Pythons :mod:`subprocess` modul
.. code-block:: python
from django_q.tasks import async, result
from django_q.tasks import enqueue, result
# make a backup copy of setup.py
async('subprocess.call', ['cp', 'setup.py', 'setup.py.bak'])
enqueue('subprocess.call', ['cp', 'setup.py', 'setup.py.bak'])
# call ls -l and dump the output
task_id=async('subprocess.check_output', ['ls', '-l'])
task_id=enqueue('subprocess.check_output', ['ls', '-l'])
# get the result
dir_list = result(task_id)
@@ -202,10 +202,10 @@ In Python 3.5 the subprocess module has changed quite a bit and returns a :class
.. code-block:: python
from django_q.tasks import async, result
from django_q.tasks import enqueue, result
# make a backup copy of setup.py
tid = async('subprocess.run', ['cp', 'setup.py', 'setup.py.bak'])
tid = enqueue('subprocess.run', ['cp', 'setup.py', 'setup.py.bak'])
# get the result
r=result(tid, 500)
@@ -220,22 +220,22 @@ In Python 3.5 the subprocess module has changed quite a bit and returns a :class
from subprocess import PIPE
# call ls -l and pipe the output
tid = async('subprocess.run', ['ls', '-l'], stdout=PIPE)
tid = enqueue('subprocess.run', ['ls', '-l'], stdout=PIPE)
# get the result
res = result(tid, 500)
# print the output
print(res.stdout)
Instead of :func:`async` you can of course also use :func:`schedule` to schedule commands.
Instead of :func:`enqueue` you can of course also use :func:`schedule` to schedule commands.
For regular Django management commands, it is easier to call them directly:
.. code-block:: python
from django_q.tasks import async, schedule
from django_q.tasks import enqueue, schedule
async('django.core.management.call_command','clearsessions')
enqueue('django.core.management.call_command','clearsessions')
# or clear those sessions every hour
@@ -255,7 +255,7 @@ Adapted from `Sebastian Raschka's blog <http://sebastianraschka.com/Articles/201
# Group example with Parzen-window estimation
import numpy
from django_q.tasks import async, result_group, delete_group
from django_q.tasks import enqueue, result_group, delete_group
# the estimation function
def parzen_estimation(x_samples, point_x, h):
@@ -270,7 +270,7 @@ Adapted from `Sebastian Raschka's blog <http://sebastianraschka.com/Articles/201
return h, (k_n / len(x_samples)) / (h ** point_x.shape[1])
# create 100 calculations and return the collated result
def parzen_async():
def parzen_enqueue():
# clear the previous results
delete_group('parzen', cached=True)
mu_vec = numpy.array([0, 0])
@@ -279,10 +279,10 @@ Adapted from `Sebastian Raschka's blog <http://sebastianraschka.com/Articles/201
multivariate_normal(mu_vec, cov_mat, 10000)
widths = numpy.linspace(1.0, 1.2, 100)
x = numpy.array([[0], [0]])
# async them with a group label to the cache backend
# enqueue them with a group label to the cache backend
for w in widths:
async(parzen_estimation, sample, x, w,
group='parzen', cached=True)
enqueue(parzen_estimation, sample, x, w,
group='parzen', cached=True)
# return after 100 results
return result_group('parzen', count=100, cached=True)
@@ -290,21 +290,21 @@ Adapted from `Sebastian Raschka's blog <http://sebastianraschka.com/Articles/201
Django Q is not optimized for distributed computing, but this example will give you an idea of what you can do with task :doc:`group`.
Alternatively the ``parzen_async()`` function can also be written with :func:`async_iter`, which automatically utilizes the cache backend and groups to return a single result from an iterable:
Alternatively the ``parzen_enqueue()`` function can also be written with :func:`enqueue_iter`, which automatically utilizes the cache backend and groups to return a single result from an iterable:
.. code-block:: python
# create 100 calculations and return the collated result
def parzen_async():
def parzen_enqueue():
mu_vec = numpy.array([0, 0])
cov_mat = numpy.array([[1, 0], [0, 1]])
sample = numpy.random. \
multivariate_normal(mu_vec, cov_mat, 10000)
widths = numpy.linspace(1.0, 1.2, 100)
x = numpy.array([[0], [0]])
# async them with async iterable
# enqueue them with enqueue iterable
args = [(sample, x, w) for w in widths]
result_id = async_iter(parzen_estimation, args, cached=True)
result_id = enqueue_iter(parzen_estimation, args, cached=True)
# return the cached result or timeout after 10 seconds
return result(result_id, wait=10000, cached=True)