mirror of
https://github.com/django-q2/django-q2.git
synced 2026-10-01 04:38:11 +08:00
Remove arrow dependency, update packages, set up testing with compose (#17)
This commit is contained in:
Stan Triepels
+1
-17
@@ -2,7 +2,7 @@ Brokers
|
||||
=======
|
||||
|
||||
The broker sits between your Django instances and your Django Q2 cluster instances; accepting, saving and delivering task packages.
|
||||
Currently we support a variety of brokers from the default Redis, bleeding edge Disque to the convenient ORM and fast MongoDB.
|
||||
Currently we support a variety of brokers.
|
||||
|
||||
The default Redis broker does not support message receipts.
|
||||
This means that in case of a catastrophic failure of the cluster server or worker timeouts, tasks that were being executed get lost.
|
||||
@@ -38,22 +38,6 @@ The default broker for Django Q2 clusters.
|
||||
* Can use existing :ref:`django_redis` connections.
|
||||
* Configure with :ref:`redis_configuration`-py compatible configuration
|
||||
|
||||
Disque
|
||||
------
|
||||
Unlike Redis, Disque supports message receipts which make delivery to the cluster workers guaranteed.
|
||||
In our tests it is as fast or faster than the Redis broker.
|
||||
You can control the amount of time Disque should wait for completion of a task by configuring the :ref:`retry` setting.
|
||||
Bulk task retrieval is supported via the :ref:`bulk` option.
|
||||
|
||||
* Delivery receipts
|
||||
* Atomic
|
||||
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/4.0/topics/cache/#setting-up-the-cache>`__ configured for monitoring
|
||||
* Compatible with `Tynd <https://disque.tynd.co/>`__ Disque addon on `Heroku <https://heroku.com>`__
|
||||
* Still considered Alpha software
|
||||
* Supports bulk dequeue
|
||||
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
|
||||
* See the :ref:`disque_configuration` configuration section for more info.
|
||||
|
||||
IronMQ
|
||||
------
|
||||
This HTTP based queue service is both available directly via `Iron.io <http://www.iron.io/mq/>`__ and as an add-on on Heroku.
|
||||
|
||||
@@ -235,43 +235,6 @@ of the cache connection you want to use instead of a direct Redis connection::
|
||||
.. tip::
|
||||
Django Q2 uses your ``SECRET_KEY`` to sign task packages and prevent task crossover. So make sure you have it set up in your Django settings.
|
||||
|
||||
.. _disque_configuration:
|
||||
|
||||
disque_nodes
|
||||
~~~~~~~~~~~~
|
||||
If you want to use Disque as your broker, set this to a list of available Disque nodes and each cluster will randomly try to connect to them::
|
||||
|
||||
# example disque connection
|
||||
Q_CLUSTER = {
|
||||
'name': 'DisqueBroker',
|
||||
'workers': 4,
|
||||
'timeout': 60,
|
||||
'retry': 60,
|
||||
'disque_nodes': ['127.0.0.1:7711', '127.0.0.1:7712']
|
||||
}
|
||||
|
||||
|
||||
Django Q2 is also compatible with the `Tynd Disque <https://disque.tynd.co/>`__ addon on `Heroku <https://heroku.com>`__::
|
||||
|
||||
# example Tynd Disque connection
|
||||
import os
|
||||
|
||||
Q_CLUSTER = {
|
||||
'name': 'TyndBroker',
|
||||
'workers': 8,
|
||||
'timeout': 30,
|
||||
'retry': 60,
|
||||
'bulk': 10,
|
||||
'disque_nodes': os.environ['TYND_DISQUE_NODES'].split(','),
|
||||
'disque_auth': os.environ['TYND_DISQUE_AUTH']
|
||||
}
|
||||
|
||||
|
||||
disque_auth
|
||||
~~~~~~~~~~~
|
||||
|
||||
Optional Disque password for servers that require authentication.
|
||||
|
||||
.. _ironmq_configuration:
|
||||
|
||||
iron_mq
|
||||
|
||||
+1
-1
@@ -23,7 +23,7 @@ Features
|
||||
- Django Admin integration
|
||||
- PaaS compatible with multiple instances
|
||||
- Multi cluster monitor
|
||||
- Redis, Disque, IronMQ, SQS, MongoDB or ORM
|
||||
- Redis, IronMQ, SQS, MongoDB or ORM
|
||||
- Rollbar and Sentry support
|
||||
|
||||
|
||||
|
||||
+3
-9
@@ -32,16 +32,12 @@ Django Q2 is tested for Python 3.7, 3.8, 3.9 and 3.10
|
||||
- `Django <https://www.djangoproject.com>`__
|
||||
|
||||
Django Q2 aims to use as much of Django's standard offerings as possible.
|
||||
The code is tested against Django versions `2.2.x` and `3.2.x`.
|
||||
The code is tested against Django versions `3.2.x` and `4.1.x`.
|
||||
|
||||
- `Django-picklefield <https://github.com/gintas/django-picklefield>`__
|
||||
|
||||
Used to store args, kwargs and result objects in the database.
|
||||
|
||||
- `Arrow <https://github.com/crsmithdev/arrow>`__
|
||||
|
||||
The scheduler uses Chris Smith's wonderful project to determine correct dates in the future.
|
||||
|
||||
- `Blessed <https://github.com/jquast/blessed>`__
|
||||
|
||||
This feature-filled fork of Erik Rose's blessings project provides the terminal layout of the monitor.
|
||||
@@ -49,7 +45,7 @@ Django Q2 is tested for Python 3.7, 3.8, 3.9 and 3.10
|
||||
|
||||
Optional
|
||||
~~~~~~~~
|
||||
- `Redis-py <https://github.com/andymccurdy/redis-py>`__ client by Andy McCurdy is used to interface with both the Redis and Disque brokers::
|
||||
- `Redis-py <https://github.com/andymccurdy/redis-py>`__ client by Andy McCurdy is used to interface with both the Redis::
|
||||
|
||||
$ pip install redis
|
||||
|
||||
@@ -77,8 +73,6 @@ Optional
|
||||
|
||||
- `Redis <http://redis.io/>`__ server is the default broker for Django Q2. It provides the best performance and does not require Django's cache framework for monitoring.
|
||||
|
||||
- `Disque <https://github.com/antirez/disque>`__ server is based on Redis by the same author, but focuses on reliable queues. Currently in Alpha, but highly recommended. You can either build it from source or use it on Heroku through the `Tynd <https://disque.tynd.co/>`__ beta.
|
||||
|
||||
- `MongoDB <https://www.mongodb.org/>`__ is a highly scalable NoSQL database which makes for a very fast and reliably persistent at-least-once message broker. Usually available on most PaaS providers.
|
||||
|
||||
- `Pyrollbar <https://github.com/rollbar/pyrollbar>`__ is an error notifier for `Rollbar <https://rollbar.com/>`__ which lets you manage your worker errors in one place. Needs a `Rollbar <https://rollbar.com/>`__ account and access key::
|
||||
@@ -142,7 +136,7 @@ You can reference the `requirements <https://github.com/GDay/django-q2/blob/mast
|
||||
Django
|
||||
~~~~~~
|
||||
We strive to be compatible with last two major version of Django.
|
||||
At the moment this means we support the 2.2.x and 3.2.x releases.
|
||||
At the moment this means we support the 3.2.x and 4.1.x releases.
|
||||
|
||||
Since we are now no longer supporting Python 2, we can also not support older versions of Django that do not support Python >= 3.6
|
||||
For this you can always use older releases, but they are no longer maintained.
|
||||
|
||||
+2
-2
@@ -37,14 +37,14 @@ You can manage them through the :ref:`admin_page` or directly from your code wit
|
||||
|
||||
# Run a schedule every 5 minutes, starting at 6 today
|
||||
# for 2 hours
|
||||
import arrow
|
||||
from datetime import datetime
|
||||
|
||||
schedule('math.hypot',
|
||||
3, 4,
|
||||
schedule_type=Schedule.MINUTES,
|
||||
minutes=5,
|
||||
repeats=24,
|
||||
next_run=arrow.utcnow().replace(hour=18, minute=0))
|
||||
next_run=datetime.utcnow().replace(hour=18, minute=0))
|
||||
|
||||
# Use a cron expression
|
||||
schedule('math.hypot',
|
||||
|
||||
Reference in New Issue
Block a user