From 5f4334d85ad84035b338d548e5eb209926a420eb Mon Sep 17 00:00:00 2001 From: Ilan Steemers Date: Tue, 8 Sep 2015 17:17:54 +0200 Subject: [PATCH] docs: adds IronMQ and bulk config --- README.rst | 5 ++--- docs/brokers.rst | 20 +++++++++++++++----- docs/configure.rst | 34 ++++++++++++++++++++++++++++++++++ docs/index.rst | 2 +- 4 files changed, 52 insertions(+), 9 deletions(-) diff --git a/README.rst b/README.rst index 0909c8d..af1cfa5 100644 --- a/README.rst +++ b/README.rst @@ -20,7 +20,7 @@ Features - Django Admin integration - PaaS compatible with multiple instances - Multi cluster monitor -- Redis broker and Disque broker +- Redis, Disque or IronMQ broker - Python 2 and 3 Requirements @@ -37,8 +37,7 @@ Brokers ~~~~~~~ - `Redis `__ - `Disque `__ -- `Amazon SQS `__ (TBA) -- `IronMQ `__ (TBA) +- `IronMQ `__ Installation diff --git a/docs/brokers.rst b/docs/brokers.rst index c285e15..a062e06 100644 --- a/docs/brokers.rst +++ b/docs/brokers.rst @@ -2,9 +2,8 @@ Brokers ======= The broker sits between your Django instances and your Django Q cluster instances, accepting and delivering task packages. -Currently we only support `Redis `__ and `Disque `__, but support for other brokers is being worked on. - -Clients for `Amazon SQS `__ and `IronMQ `__ are TBA. +Currently we support `Redis `__ , `Disque `__ and `IronMQ `__. +Support for more brokers is being worked on. Redis @@ -12,9 +11,9 @@ Redis The default broker for Django Q clusters. * Atomic -* Does not need separate cache framework for monitoring -* Does not support receipts * Requires `Redis-py `__ client library: ``pip install redis`` +* Does not need cache framework for monitoring +* Does not support receipts * Can use existing :ref:`django_redis` connections. * Configure with :ref:`redis_configuration`-py compatible configuration @@ -28,9 +27,20 @@ You can control the amount of time Disque should wait for completion of a task b * Needs Django's `Cache framework `__ configured for monitoring * Compatible with `Tynd `__ Disque addon on `Heroku `__ * Still considered Alpha software +* Supports bulk dequeue * Requires `Redis-py `__ client library: ``pip install redis`` * See the :ref:`disque_configuration` configuration section for more info. +IronMQ +------ +This HTTP based queue service is both available directly via `Iron.io `__ and as an add-on on Heroku. + +* Delivery receipts +* Supports bulk dequeue +* Needs Django's `Cache framework `__ configured for monitoring +* Requires the `iron-mq `__ client library: ``pip install iron-mq`` +* See the :ref:`ironmq_configuration` configuration section for options. + Reference --------- The :class:`Broker` class is used internally to communicate with the different types of brokers. diff --git a/docs/configure.rst b/docs/configure.rst index 423af59..dfc34f9 100644 --- a/docs/configure.rst +++ b/docs/configure.rst @@ -187,6 +187,40 @@ disque_auth Optional Disque password for servers that require authentication. +.. _ironmq_configuration: + +iron_mq +~~~~~~~ +Connection settings for IronMQ:: + + # example IronMQ connection + + Q_CLUSTER = { + 'name': 'IronBroker', + 'workers': 8, + 'timeout': 30, + 'retry': 60, + 'queue_limit': 50, + 'bulk': 10, + 'iron_mq': { + 'host': 'mq-aws-us-east-1.iron.io', + 'token': 'Et1En7.....0LuW39Q', + 'project_id': '500f7b....b0f302e9' + } + } + + +All connection keywords are supported. See the `iron-mq `__ library for more info + +bulk +~~~~ +Sets the number of messages each cluster tries to get from the broker per call. Setting this on supported brokers can improve performance. +Especially HTTP based or very high latency servers can benefit from bulk dequeue. +Keep in mind however that settings this too high can degrade performance with multiple clusters or very large task packages. + +Not supported by the default Redis broker. +Defaults to 1. + cpu_affinity ~~~~~~~~~~~~ diff --git a/docs/index.rst b/docs/index.rst index 5d38a22..fb698c7 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -20,7 +20,7 @@ Features - Django Admin integration - PaaS compatible with multiple instances - Multi cluster monitor -- Redis or Disque broker +- Redis, Disque or IronMQ broker - Python 2 and 3