mirror of
https://github.com/django-q2/django-q2.git
synced 2026-09-21 00:28:12 +08:00
167 lines
3.8 KiB
ReStructuredText
167 lines
3.8 KiB
ReStructuredText
Django Q
|
|
========
|
|
|
|
A multiprocessing task queue for Django
|
|
---------------------------------------
|
|
|
|
|image0|
|
|
|
|
Features
|
|
~~~~~~~~
|
|
|
|
- Multiprocessing worker pool
|
|
- Encrypted and compressed task packages
|
|
- Scheduled tasks
|
|
- Result hooks
|
|
- Result and Failure database
|
|
- PaaS compatible with multiple pools
|
|
- Django Admin
|
|
|
|
Requirements
|
|
~~~~~~~~~~~~
|
|
|
|
- `Redis-py <https://github.com/andymccurdy/redis-py>`__
|
|
- `Django <https://www.djangoproject.com>`__ > = 1.7
|
|
- `Django-picklefield <https://github.com/gintas/django-picklefield>`__
|
|
- `Arrow <https://github.com/crsmithdev/arrow>`__
|
|
- `Blessed <https://github.com/jquast/blessed>`__
|
|
|
|
Tested with: Python 2.7, 3.4. Django 1.7.8, 1.8.2\*
|
|
|
|
\*\ *Django Q is currently in Alpha and as such not safe for production,
|
|
yet.*
|
|
|
|
Installation
|
|
~~~~~~~~~~~~
|
|
|
|
- Install the latest version with pip: ``pip install django-q``
|
|
- Add ``django_q`` to ``INSTALLED_APPS`` in your settings.py:
|
|
|
|
.. code:: python
|
|
|
|
INSTALLED_APPS = (
|
|
# other apps
|
|
'django_q',
|
|
)
|
|
|
|
- Run ``python manage.py migrate`` to create the database tables
|
|
- Make sure you have a `Redis <http://redis.io/>`__ server running
|
|
somewhere
|
|
|
|
Configuration
|
|
~~~~~~~~~~~~~
|
|
|
|
All configuration settings are optional. e.g:
|
|
|
|
.. code:: python
|
|
|
|
# settings.py
|
|
Q_CLUSTER = {
|
|
'name': 'myproject',
|
|
'workers': 8,
|
|
'recycle': 500,
|
|
'compress': True,
|
|
'save_limit': 250,
|
|
'label': 'Django Q',
|
|
'redis': {
|
|
'host': '127.0.0.1',
|
|
'port': 6379,
|
|
'db': 0, }
|
|
}
|
|
|
|
- **name** Used to differentiatie between projects using the same Redis
|
|
server\* *default*: 'default'
|
|
|
|
- **workers** The number of workers to use in the cluster *default*:
|
|
CPU count of host
|
|
|
|
- **recycle** The number of tasks a worker will process before
|
|
respawing. Used to release resources. *default*: 500
|
|
|
|
- **compress** Compress task packages to Redis. Useful for large
|
|
payloads. *default*: False
|
|
|
|
- **save\_limit** Limits the amount of successful tasks saved to
|
|
Django. Set to 0 for unlimited. Set to -1 for no success storage at
|
|
all. Failures are always saved. *default*: 250
|
|
|
|
- **label** The label used for the Django Admin page *default*: 'Django
|
|
Q'
|
|
|
|
- **redis** Connection settings for Redis. Follows standard Redis-Py
|
|
syntax. *default*: localhost
|
|
|
|
\*\ *Django Q uses your SECRET\_KEY to encrypt task packages and prevent
|
|
task crossover*
|
|
|
|
Managment Commands
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
qcluster
|
|
^^^^^^^^
|
|
|
|
Start a cluster with: ``python manage.py qcluster`` ####qmonitor Monitor
|
|
your clusters with ``python manage.py qmonitor``
|
|
|
|
Creating Tasks
|
|
~~~~~~~~~~~~~~
|
|
|
|
Async
|
|
^^^^^
|
|
|
|
.. code:: python
|
|
|
|
async(func,*args,hook=None,**kwargs)
|
|
|
|
.. code:: python
|
|
|
|
from django_q import async, result
|
|
|
|
# create the task
|
|
async('math.copysign', 2, -2)
|
|
|
|
# or with import and storing the id
|
|
import math.copysign
|
|
|
|
task_id = async(copysign, 2, -2)
|
|
|
|
# get the result
|
|
task_result = result(task_id)
|
|
|
|
# result returns None if the task has not been executed yet
|
|
# so in most cases you will want to use a hook:
|
|
|
|
async('math.modf', 2.5, hook='hooks.print_result')
|
|
|
|
# hooks.py
|
|
def print_result(task):
|
|
print(task.result)
|
|
|
|
Schedule
|
|
^^^^^^^^
|
|
|
|
Schedules are regular Django models. You can manage them through the
|
|
Admin page or in your code:
|
|
|
|
.. code:: python
|
|
|
|
from django_q import Schedule
|
|
from django.utils import timezone
|
|
|
|
Schedule.create(func='math.copysign',
|
|
hook='hooks.print_result',
|
|
args='2,-2',
|
|
schedule_type=Schedule.DAILY,
|
|
next_run=timezone.now())
|
|
|
|
Todo
|
|
----
|
|
|
|
- Write sphinx documentation
|
|
- Better tests and coverage
|
|
- Get out of Alpha
|
|
- Less dependencies?
|
|
|
|
.. |image0| image:: https://travis-ci.org/Koed00/django-q.svg?branch=master
|
|
:target: https://travis-ci.org/Koed00/django-q
|