docs: adds IronMQ and bulk config

This commit is contained in:
Ilan Steemers
2015-09-08 17:17:54 +02:00
parent 69df45ea4e
commit 5f4334d85a
4 changed files with 52 additions and 9 deletions
+2 -3
View File
@@ -20,7 +20,7 @@ Features
- Django Admin integration
- PaaS compatible with multiple instances
- Multi cluster monitor
- Redis broker and Disque broker
- Redis, Disque or IronMQ broker
- Python 2 and 3
Requirements
@@ -37,8 +37,7 @@ Brokers
~~~~~~~
- `Redis <http://redis.io/>`__
- `Disque <https://github.com/antirez/disque>`__
- `Amazon SQS <https://aws.amazon.com/sqs/>`__ (TBA)
- `IronMQ <http://www.iron.io/mq/>`__ (TBA)
- `IronMQ <http://www.iron.io/mq/>`__
Installation
+15 -5
View File
@@ -2,9 +2,8 @@ Brokers
=======
The broker sits between your Django instances and your Django Q cluster instances, accepting and delivering task packages.
Currently we only support `Redis <http://redis.io/>`__ and `Disque <https://github.com/antirez/disque>`__, but support for other brokers is being worked on.
Clients for `Amazon SQS <https://aws.amazon.com/sqs/>`__ and `IronMQ <http://www.iron.io/mq/>`__ are TBA.
Currently we support `Redis <http://redis.io/>`__ , `Disque <https://github.com/antirez/disque>`__ and `IronMQ <http://www.iron.io/mq/>`__.
Support for more brokers is being worked on.
Redis
@@ -12,9 +11,9 @@ Redis
The default broker for Django Q clusters.
* Atomic
* Does not need separate cache framework for monitoring
* Does not support receipts
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
* Does not need cache framework for monitoring
* Does not support receipts
* Can use existing :ref:`django_redis` connections.
* Configure with :ref:`redis_configuration`-py compatible configuration
@@ -28,9 +27,20 @@ You can control the amount of time Disque should wait for completion of a task b
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
* Compatible with `Tynd <https://disque.tynd.co/>`__ Disque addon on `Heroku <https://heroku.com>`__
* Still considered Alpha software
* Supports bulk dequeue
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
* See the :ref:`disque_configuration` configuration section for more info.
IronMQ
------
This HTTP based queue service is both available directly via `Iron.io <http://www.iron.io/mq/>`__ and as an add-on on Heroku.
* Delivery receipts
* Supports bulk dequeue
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
* Requires the `iron-mq <https://github.com/iron-io/iron_mq_python>`__ client library: ``pip install iron-mq``
* See the :ref:`ironmq_configuration` configuration section for options.
Reference
---------
The :class:`Broker` class is used internally to communicate with the different types of brokers.
+34
View File
@@ -187,6 +187,40 @@ disque_auth
Optional Disque password for servers that require authentication.
.. _ironmq_configuration:
iron_mq
~~~~~~~
Connection settings for IronMQ::
# example IronMQ connection
Q_CLUSTER = {
'name': 'IronBroker',
'workers': 8,
'timeout': 30,
'retry': 60,
'queue_limit': 50,
'bulk': 10,
'iron_mq': {
'host': 'mq-aws-us-east-1.iron.io',
'token': 'Et1En7.....0LuW39Q',
'project_id': '500f7b....b0f302e9'
}
}
All connection keywords are supported. See the `iron-mq <https://github.com/iron-io/iron_mq_python#configure>`__ library for more info
bulk
~~~~
Sets the number of messages each cluster tries to get from the broker per call. Setting this on supported brokers can improve performance.
Especially HTTP based or very high latency servers can benefit from bulk dequeue.
Keep in mind however that settings this too high can degrade performance with multiple clusters or very large task packages.
Not supported by the default Redis broker.
Defaults to 1.
cpu_affinity
~~~~~~~~~~~~
+1 -1
View File
@@ -20,7 +20,7 @@ Features
- Django Admin integration
- PaaS compatible with multiple instances
- Multi cluster monitor
- Redis or Disque broker
- Redis, Disque or IronMQ broker
- Python 2 and 3