diff --git a/README.md b/README.md index 2d78362..2217bee 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,19 @@ # Django Q ##A multiprocessing task queue application for Django ### Status -Currently in the pre-alpha stage and Python 3 only (for now). +In Alpha and Python 3 only (for now). +Everything should work, but the basic structure can still change. +Main focus will be put on creating tests now. + ### Architecture ![Django Q schema](http://i.imgur.com/wTIeg2T.png) + + +```python +async(func,*args,hook=None,**kwargs) +``` + ### Signed Tasks Tasks are first pickled to Json and then signed using Django's own signing module before being sent to a Redis list. This ensures that task packages on the Redis server can only be excuted and read by clusters and django servers who share the same secret key. @@ -17,7 +26,7 @@ The pusher process continously checks the Redis list for new task packages and p A worker process checks the package signing, unpacks the task, executes it and saves the return value. Irrespective of the failure or success of any of these steps, the package is then pushed onto the Result Queue. By default Django Q spawns a worker for each detected CPU on the host system. -This can be overridden by setting `Q_WORKERS = n`. With *n* being the numbered of desired worker processes. +This can be overridden by setting `Q_WORKERS = n`. With *n* being the numbe of desired worker processes. ### Monitor The result monitor checks the Result Queue for processed packages and saves both failed and succesful packages to the Django database. @@ -36,6 +45,12 @@ In case of a stop signal, the sentinel will halt the pusher and instruct the wor Packages can be assigned a hook function, upon completion of the package this function will be called with the Task object as the first argument. +### Management command +Start the cluster with `./manage.py qcluster` + +###Admin integration +Django Q registers itself with the admin page to show failed and succesful tasks. +From there task results can be read or deleted. If neccesary, failed tasks can be reintroduced to the queue. ### Todo I'll add to this README while I'm developing the various parts. \ No newline at end of file diff --git a/README.rst b/README.rst index c9da216..4d1a4ef 100644 --- a/README.rst +++ b/README.rst @@ -7,16 +7,28 @@ A multiprocessing task queue application for Django Status ~~~~~~ -Currently in the pre-alpha stage and Python 3 only (for now). +In Alpha and Python 3 only (for now). Everything should work, but the +basic structure can still change. Main focus will be put on creating +tests now. Architecture ~~~~~~~~~~~~ -|Django Q schema| ### Signed Tasks Tasks are first pickled to Json and -then signed using Django's own signing module before being sent to a -Redis list. This ensures that task packages on the Redis server can only -be excuted and read by clusters and django servers who share the same -secret key. +.. figure:: http://i.imgur.com/wTIeg2T.png + :alt: Django Q schema + + Django Q schema +.. code:: python + + async(func,*args,hook=None,**kwargs) + +Signed Tasks +~~~~~~~~~~~~ + +Tasks are first pickled to Json and then signed using Django's own +signing module before being sent to a Redis list. This ensures that task +packages on the Redis server can only be excuted and read by clusters +and django servers who share the same secret key. Optionally, packages can be compressed before transport by setting ``Q_COMPRESSED = True`` @@ -36,7 +48,7 @@ any of these steps, the package is then pushed onto the Result Queue. By default Django Q spawns a worker for each detected CPU on the host system. This can be overridden by setting ``Q_WORKERS = n``. With *n* -being the numbered of desired worker processes. +being the numbe of desired worker processes. Monitor ~~~~~~~ @@ -65,9 +77,19 @@ Hooks Packages can be assigned a hook function, upon completion of the package this function will be called with the Task object as the first argument. +Management command +~~~~~~~~~~~~~~~~~~ + +Start the cluster with ``./manage.py qcluster`` + +Admin integration +~~~~~~~~~~~~~~~~~ + +Django Q registers itself with the admin page to show failed and +succesful tasks. From there task results can be read or deleted. If +neccesary, failed tasks can be reintroduced to the queue. + Todo ~~~~ I'll add to this README while I'm developing the various parts. - -.. |Django Q schema| image:: http://i.imgur.com/wTIeg2T.png diff --git a/django_q/tests/tasks.py b/django_q/tests/tasks.py index fc1ecf8..4ac10a1 100644 --- a/django_q/tests/tasks.py +++ b/django_q/tests/tasks.py @@ -9,5 +9,16 @@ def countdown(n): return stop_time - start_time +def count_letters(tup): + total = 0 + for word in tup: + total += len(word) + return total + +def count_letters2(obj): + return count_letters(obj.get_words()) + def result(obj): - print('RESULT HOOK: {}'.format(obj.result)) + print('RESULT HOOK {} : {}'.format(obj.name, obj.result)) + +