docs: adds IronMQ and bulk config

This commit is contained in:
Ilan Steemers
2015-09-08 17:17:54 +02:00
parent 69df45ea4e
commit 5f4334d85a
4 changed files with 52 additions and 9 deletions
+2 -3
View File
@@ -20,7 +20,7 @@ Features
- Django Admin integration - Django Admin integration
- PaaS compatible with multiple instances - PaaS compatible with multiple instances
- Multi cluster monitor - Multi cluster monitor
- Redis broker and Disque broker - Redis, Disque or IronMQ broker
- Python 2 and 3 - Python 2 and 3
Requirements Requirements
@@ -37,8 +37,7 @@ Brokers
~~~~~~~ ~~~~~~~
- `Redis <http://redis.io/>`__ - `Redis <http://redis.io/>`__
- `Disque <https://github.com/antirez/disque>`__ - `Disque <https://github.com/antirez/disque>`__
- `Amazon SQS <https://aws.amazon.com/sqs/>`__ (TBA) - `IronMQ <http://www.iron.io/mq/>`__
- `IronMQ <http://www.iron.io/mq/>`__ (TBA)
Installation Installation
+15 -5
View File
@@ -2,9 +2,8 @@ Brokers
======= =======
The broker sits between your Django instances and your Django Q cluster instances, accepting and delivering task packages. The broker sits between your Django instances and your Django Q cluster instances, accepting and delivering task packages.
Currently we only support `Redis <http://redis.io/>`__ and `Disque <https://github.com/antirez/disque>`__, but support for other brokers is being worked on. Currently we support `Redis <http://redis.io/>`__ , `Disque <https://github.com/antirez/disque>`__ and `IronMQ <http://www.iron.io/mq/>`__.
Support for more brokers is being worked on.
Clients for `Amazon SQS <https://aws.amazon.com/sqs/>`__ and `IronMQ <http://www.iron.io/mq/>`__ are TBA.
Redis Redis
@@ -12,9 +11,9 @@ Redis
The default broker for Django Q clusters. The default broker for Django Q clusters.
* Atomic * Atomic
* Does not need separate cache framework for monitoring
* Does not support receipts
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis`` * Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
* Does not need cache framework for monitoring
* Does not support receipts
* Can use existing :ref:`django_redis` connections. * Can use existing :ref:`django_redis` connections.
* Configure with :ref:`redis_configuration`-py compatible configuration * Configure with :ref:`redis_configuration`-py compatible configuration
@@ -28,9 +27,20 @@ You can control the amount of time Disque should wait for completion of a task b
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring * Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
* Compatible with `Tynd <https://disque.tynd.co/>`__ Disque addon on `Heroku <https://heroku.com>`__ * Compatible with `Tynd <https://disque.tynd.co/>`__ Disque addon on `Heroku <https://heroku.com>`__
* Still considered Alpha software * Still considered Alpha software
* Supports bulk dequeue
* Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis`` * Requires `Redis-py <https://github.com/andymccurdy/redis-py>`__ client library: ``pip install redis``
* See the :ref:`disque_configuration` configuration section for more info. * See the :ref:`disque_configuration` configuration section for more info.
IronMQ
------
This HTTP based queue service is both available directly via `Iron.io <http://www.iron.io/mq/>`__ and as an add-on on Heroku.
* Delivery receipts
* Supports bulk dequeue
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
* Requires the `iron-mq <https://github.com/iron-io/iron_mq_python>`__ client library: ``pip install iron-mq``
* See the :ref:`ironmq_configuration` configuration section for options.
Reference Reference
--------- ---------
The :class:`Broker` class is used internally to communicate with the different types of brokers. The :class:`Broker` class is used internally to communicate with the different types of brokers.
+34
View File
@@ -187,6 +187,40 @@ disque_auth
Optional Disque password for servers that require authentication. Optional Disque password for servers that require authentication.
.. _ironmq_configuration:
iron_mq
~~~~~~~
Connection settings for IronMQ::
# example IronMQ connection
Q_CLUSTER = {
'name': 'IronBroker',
'workers': 8,
'timeout': 30,
'retry': 60,
'queue_limit': 50,
'bulk': 10,
'iron_mq': {
'host': 'mq-aws-us-east-1.iron.io',
'token': 'Et1En7.....0LuW39Q',
'project_id': '500f7b....b0f302e9'
}
}
All connection keywords are supported. See the `iron-mq <https://github.com/iron-io/iron_mq_python#configure>`__ library for more info
bulk
~~~~
Sets the number of messages each cluster tries to get from the broker per call. Setting this on supported brokers can improve performance.
Especially HTTP based or very high latency servers can benefit from bulk dequeue.
Keep in mind however that settings this too high can degrade performance with multiple clusters or very large task packages.
Not supported by the default Redis broker.
Defaults to 1.
cpu_affinity cpu_affinity
~~~~~~~~~~~~ ~~~~~~~~~~~~
+1 -1
View File
@@ -20,7 +20,7 @@ Features
- Django Admin integration - Django Admin integration
- PaaS compatible with multiple instances - PaaS compatible with multiple instances
- Multi cluster monitor - Multi cluster monitor
- Redis or Disque broker - Redis, Disque or IronMQ broker
- Python 2 and 3 - Python 2 and 3