mirror of
https://github.com/django-q2/django-q2.git
synced 2026-10-08 13:38:12 +08:00
docs: adds async_iter
This commit is contained in:
+44
-3
@@ -89,6 +89,28 @@ Please not that this will override any other option keywords.
|
|||||||
For tasks to be processed you will need to have a worker cluster running in the background using ``python manage.py qcluster``
|
For tasks to be processed you will need to have a worker cluster running in the background using ``python manage.py qcluster``
|
||||||
or you need to configure Django Q to run in synchronous mode for testing using the :ref:`sync` option.
|
or you need to configure Django Q to run in synchronous mode for testing using the :ref:`sync` option.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Async Iterable
|
||||||
|
--------------
|
||||||
|
If you have an iterable object with arguments for a function, you can use :func:`async_iter` to async them with a single command::
|
||||||
|
|
||||||
|
# Async Iterable example
|
||||||
|
from django_q.tasks import async_iter, result
|
||||||
|
|
||||||
|
# set up a list of arguments for math.floor
|
||||||
|
iter = [i for i in range(100)]
|
||||||
|
|
||||||
|
# async iter them
|
||||||
|
id=async_iter('math.floor',iter)
|
||||||
|
|
||||||
|
# wait for the collated result for 1 second
|
||||||
|
result_list = result(id, wait=1000)
|
||||||
|
|
||||||
|
This will individually queue 100 tasks to the worker cluster, which will save their results in the cache backend for speed.
|
||||||
|
Once all the 100 results are in the cache, they are collated into a list and saved as a single result in the database. The cache results are then cleared.
|
||||||
|
Needs the Django cache framework.
|
||||||
|
|
||||||
.. _groups:
|
.. _groups:
|
||||||
|
|
||||||
Groups
|
Groups
|
||||||
@@ -103,14 +125,16 @@ You can group together results by passing :func:`async` the optional ``group`` k
|
|||||||
for i in range(4):
|
for i in range(4):
|
||||||
async('math.modf', i, group='modf')
|
async('math.modf', i, group='modf')
|
||||||
|
|
||||||
# after the tasks have finished you can get the group results
|
# wait until the group has 4 results
|
||||||
result = result_group('modf')
|
result = result_group('modf', count=4)
|
||||||
print(result)
|
print(result)
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
[(0.0, 0.0), (0.0, 1.0), (0.0, 2.0), (0.0, 3.0)]
|
[(0.0, 0.0), (0.0, 1.0), (0.0, 2.0), (0.0, 3.0)]
|
||||||
|
|
||||||
|
Note that the same can be achieved much faster with :func:`async_iter`
|
||||||
|
|
||||||
Take care to not limit your results database too much and call :func:`delete_group` before each run, unless you want your results to keep adding up.
|
Take care to not limit your results database too much and call :func:`delete_group` before each run, unless you want your results to keep adding up.
|
||||||
Instead of :func:`result_group` you can also use :func:`fetch_group` to return a queryset of :class:`Task` objects.:
|
Instead of :func:`result_group` you can also use :func:`fetch_group` to return a queryset of :class:`Task` objects.:
|
||||||
|
|
||||||
@@ -163,7 +187,7 @@ By using a cache backend like Redis or Memcached you can speed up access to your
|
|||||||
|
|
||||||
When you set ``cached=True``, results will be saved permanently in the cache and you will have to rely on your backend's cleanup strategies (like LRU) to
|
When you set ``cached=True``, results will be saved permanently in the cache and you will have to rely on your backend's cleanup strategies (like LRU) to
|
||||||
manage stale results.
|
manage stale results.
|
||||||
You can however opt to set a manual timeout on the results, by setting ``cached=60``. Meaning the result will be evicted from the cache after 60 seconds.
|
You can also opt to set a manual timeout on the results, by setting ``cached=60``. Meaning the result will be evicted from the cache after 60 seconds.
|
||||||
This works both globally or on individual async executions.::
|
This works both globally or on individual async executions.::
|
||||||
|
|
||||||
# simple cached example
|
# simple cached example
|
||||||
@@ -175,6 +199,12 @@ This works both globally or on individual async executions.::
|
|||||||
# wait max 50ms for the result to appear in the cache
|
# wait max 50ms for the result to appear in the cache
|
||||||
result(id, wait=50, cached=True)
|
result(id, wait=50, cached=True)
|
||||||
|
|
||||||
|
# o fetch the task object
|
||||||
|
task = fetch(id, cache=True)
|
||||||
|
|
||||||
|
# and then save it to the database
|
||||||
|
task.save()
|
||||||
|
|
||||||
This also works for group actions::
|
This also works for group actions::
|
||||||
|
|
||||||
# cached group example
|
# cached group example
|
||||||
@@ -195,6 +225,7 @@ This also works for group actions::
|
|||||||
# wait max 50ms for one hundred results to return
|
# wait max 50ms for one hundred results to return
|
||||||
result_group('frexp', wait=50, count=100, cached=True)
|
result_group('frexp', wait=50, count=100, cached=True)
|
||||||
|
|
||||||
|
Note that exact same result can be achieved by using the more convenient :func:`async_iter` in this case, but without hook support.
|
||||||
|
|
||||||
Synchronous testing
|
Synchronous testing
|
||||||
-------------------
|
-------------------
|
||||||
@@ -287,6 +318,16 @@ Reference
|
|||||||
|
|
||||||
Renamed from get_task
|
Renamed from get_task
|
||||||
|
|
||||||
|
.. py:function:: async_iter(func, args_iter,**kwargs)
|
||||||
|
|
||||||
|
Runs iterable arguments against the cache backend and returns a single collated result
|
||||||
|
|
||||||
|
:param object func: The task function to execute
|
||||||
|
:param args: An iterable containing arguments for the task function
|
||||||
|
:param dict kwargs: Keyword arguments for the task function. Ignores ``cached`` and ``hook``.
|
||||||
|
:returns: The uuid of the task
|
||||||
|
:rtype: str
|
||||||
|
|
||||||
.. py:function:: queue_size()
|
.. py:function:: queue_size()
|
||||||
|
|
||||||
Returns the size of the broker queue.
|
Returns the size of the broker queue.
|
||||||
|
|||||||
Reference in New Issue
Block a user