mirror of
https://github.com/django-q2/django-q2.git
synced 2026-10-08 01:18:12 +08:00
docs: adds IronMQ and bulk config
This commit is contained in:
+2
-3
@@ -20,7 +20,7 @@ Features
|
|||||||
- Django Admin integration
|
- Django Admin integration
|
||||||
- PaaS compatible with multiple instances
|
- PaaS compatible with multiple instances
|
||||||
- Multi cluster monitor
|
- Multi cluster monitor
|
||||||
- Redis broker and Disque broker
|
- Redis, Disque or IronMQ broker
|
||||||
- Python 2 and 3
|
- Python 2 and 3
|
||||||
|
|
||||||
Requirements
|
Requirements
|
||||||
@@ -37,8 +37,7 @@ Brokers
|
|||||||
~~~~~~~
|
~~~~~~~
|
||||||
- `Redis <http://redis.io/>`__
|
- `Redis <http://redis.io/>`__
|
||||||
- `Disque <https://github.com/antirez/disque>`__
|
- `Disque <https://github.com/antirez/disque>`__
|
||||||
- `Amazon SQS <https://aws.amazon.com/sqs/>`__ (TBA)
|
- `IronMQ <http://www.iron.io/mq/>`__
|
||||||
- `IronMQ <http://www.iron.io/mq/>`__ (TBA)
|
|
||||||
|
|
||||||
|
|
||||||
Installation
|
Installation
|
||||||
|
|||||||
+15
-5
@@ -2,9 +2,8 @@ Brokers
|
|||||||
=======
|
=======
|
||||||
|
|
||||||
The broker sits between your Django instances and your Django Q cluster instances, accepting and delivering task packages.
|
The broker sits between your Django instances and your Django Q cluster instances, accepting and delivering task packages.
|
||||||
Currently we only support `Redis <http://redis.io/>`__ and `Disque <https://github.com/antirez/disque>`__, but support for other brokers is being worked on.
|
Currently we support `Redis <http://redis.io/>`__ , `Disque <https://github.com/antirez/disque>`__ and `IronMQ <http://www.iron.io/mq/>`__.
|
||||||
|
Support for more brokers is being worked on.
|
||||||
Clients for `Amazon SQS <https://aws.amazon.com/sqs/>`__ and `IronMQ <http://www.iron.io/mq/>`__ are TBA.
|
|
||||||
|
|
||||||
|
|
||||||
Redis
|
Redis
|
||||||
@@ -12,9 +11,9 @@ Redis
|
|||||||
The default broker for Django Q clusters.
|
The default broker for Django Q clusters.
|
||||||
|
|
||||||
* Atomic
|
* Atomic
|
||||||
* Does not need separate cache framework for monitoring
|
|
||||||
* Does not support receipts
|
|
||||||
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
|
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
|
||||||
|
* Does not need cache framework for monitoring
|
||||||
|
* Does not support receipts
|
||||||
* Can use existing :ref:`django_redis` connections.
|
* Can use existing :ref:`django_redis` connections.
|
||||||
* Configure with :ref:`redis_configuration`-py compatible configuration
|
* Configure with :ref:`redis_configuration`-py compatible configuration
|
||||||
|
|
||||||
@@ -28,9 +27,20 @@ You can control the amount of time Disque should wait for completion of a task b
|
|||||||
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
|
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
|
||||||
* Compatible with `Tynd <https://disque.tynd.co/>`__ Disque addon on `Heroku <https://heroku.com>`__
|
* Compatible with `Tynd <https://disque.tynd.co/>`__ Disque addon on `Heroku <https://heroku.com>`__
|
||||||
* Still considered Alpha software
|
* Still considered Alpha software
|
||||||
|
* Supports bulk dequeue
|
||||||
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
|
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
|
||||||
* See the :ref:`disque_configuration` configuration section for more info.
|
* See the :ref:`disque_configuration` configuration section for more info.
|
||||||
|
|
||||||
|
IronMQ
|
||||||
|
------
|
||||||
|
This HTTP based queue service is both available directly via `Iron.io <http://www.iron.io/mq/>`__ and as an add-on on Heroku.
|
||||||
|
|
||||||
|
* Delivery receipts
|
||||||
|
* Supports bulk dequeue
|
||||||
|
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
|
||||||
|
* Requires the `iron-mq <https://github.com/iron-io/iron_mq_python>`__ client library: ``pip install iron-mq``
|
||||||
|
* See the :ref:`ironmq_configuration` configuration section for options.
|
||||||
|
|
||||||
Reference
|
Reference
|
||||||
---------
|
---------
|
||||||
The :class:`Broker` class is used internally to communicate with the different types of brokers.
|
The :class:`Broker` class is used internally to communicate with the different types of brokers.
|
||||||
|
|||||||
@@ -187,6 +187,40 @@ disque_auth
|
|||||||
|
|
||||||
Optional Disque password for servers that require authentication.
|
Optional Disque password for servers that require authentication.
|
||||||
|
|
||||||
|
.. _ironmq_configuration:
|
||||||
|
|
||||||
|
iron_mq
|
||||||
|
~~~~~~~
|
||||||
|
Connection settings for IronMQ::
|
||||||
|
|
||||||
|
# example IronMQ connection
|
||||||
|
|
||||||
|
Q_CLUSTER = {
|
||||||
|
'name': 'IronBroker',
|
||||||
|
'workers': 8,
|
||||||
|
'timeout': 30,
|
||||||
|
'retry': 60,
|
||||||
|
'queue_limit': 50,
|
||||||
|
'bulk': 10,
|
||||||
|
'iron_mq': {
|
||||||
|
'host': 'mq-aws-us-east-1.iron.io',
|
||||||
|
'token': 'Et1En7.....0LuW39Q',
|
||||||
|
'project_id': '500f7b....b0f302e9'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
All connection keywords are supported. See the `iron-mq <https://github.com/iron-io/iron_mq_python#configure>`__ library for more info
|
||||||
|
|
||||||
|
bulk
|
||||||
|
~~~~
|
||||||
|
Sets the number of messages each cluster tries to get from the broker per call. Setting this on supported brokers can improve performance.
|
||||||
|
Especially HTTP based or very high latency servers can benefit from bulk dequeue.
|
||||||
|
Keep in mind however that settings this too high can degrade performance with multiple clusters or very large task packages.
|
||||||
|
|
||||||
|
Not supported by the default Redis broker.
|
||||||
|
Defaults to 1.
|
||||||
|
|
||||||
cpu_affinity
|
cpu_affinity
|
||||||
~~~~~~~~~~~~
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -20,7 +20,7 @@ Features
|
|||||||
- Django Admin integration
|
- Django Admin integration
|
||||||
- PaaS compatible with multiple instances
|
- PaaS compatible with multiple instances
|
||||||
- Multi cluster monitor
|
- Multi cluster monitor
|
||||||
- Redis or Disque broker
|
- Redis, Disque or IronMQ broker
|
||||||
- Python 2 and 3
|
- Python 2 and 3
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user