From 789f9342089dd37f458a3365961db8463265df7b Mon Sep 17 00:00:00 2001 From: Ilan Steemers Date: Mon, 14 Sep 2015 15:38:16 +0200 Subject: [PATCH] docs: Django ORM broker --- README.rst | 12 ++++++------ docs/admin.rst | 6 ++++++ docs/brokers.rst | 13 +++++++++++++ docs/configure.rst | 28 ++++++++++++++++++++++++++++ docs/index.rst | 2 +- 5 files changed, 54 insertions(+), 7 deletions(-) diff --git a/README.rst b/README.rst index 1a21b8b..9e0f254 100644 --- a/README.rst +++ b/README.rst @@ -20,7 +20,7 @@ Features - Django Admin integration - PaaS compatible with multiple instances - Multi cluster monitor -- Redis, Disque, IronMQ or SQS +- Redis, Disque, IronMQ, SQS or ORM - Python 2 and 3 Requirements @@ -35,11 +35,11 @@ Tested with: Python 2.7 & 3.4. Django 1.7.10 & 1.8.4 Brokers ~~~~~~~ -- `Redis `__ -- `Disque `__ -- `IronMQ `__ -- `Amazon SQS `__ - +- `Redis `__ +- `Disque `__ +- `IronMQ `__ +- `Amazon SQS `__ +- `Django ORM `__ Installation ~~~~~~~~~~~~ diff --git a/docs/admin.rst b/docs/admin.rst index fb6da85..9eace11 100644 --- a/docs/admin.rst +++ b/docs/admin.rst @@ -81,3 +81,9 @@ Indicates the success status of the last scheduled task, if any. Uses the :class:`Schedule` model + +Queued tasks +------------ +This admin view is only enabled when you use the :ref:`orm_broker` broker. +It shows all tasks packages currently in the broker queue. The ``lock`` column shows the moment at which this package was picked up by the cluster and is used to determine whether it has expired or not. +For development purposes you can edit and delete queued tasks from here. diff --git a/docs/brokers.rst b/docs/brokers.rst index 396cf0b..a3fe3e4 100644 --- a/docs/brokers.rst +++ b/docs/brokers.rst @@ -73,6 +73,19 @@ Although `SQS `__ is not the fastest, it is stable, * Requires the `boto3 `__ client library: ``pip install boto3`` * See the :ref:`sqs_configuration` configuration section for options. +.. _orm_broker: + +Django ORM +---------- +Select this to use Django's database backend as a message broker. +Unless you have configured a dedicated database backend for it, this should probably not be your first choice for a high traffic setup. +However for a medium message rate and scheduled tasks, this is the most convenient guaranteed delivery broker. + +* Delivery receipts +* Supports bulk dequeue +* Needs Django's `Cache framework `__ configured for monitoring +* Queue editable in Django Admin +* See the :ref:`orm_configuration` configuration on how to set it up. Reference --------- diff --git a/docs/configure.rst b/docs/configure.rst index 7f038fa..0f3cedc 100644 --- a/docs/configure.rst +++ b/docs/configure.rst @@ -240,6 +240,29 @@ Please make sure these credentials have proper SQS access. Amazon SQS only supports a bulk setting between 1 and 10, with the total payload not exceeding 256kb. +.. _orm_configuration: + +orm +~~~ +If you want to use Django's database backend as a message broker, set the ``orm`` keyword to the database connection you want it to use:: + + # example ORM broker connection + + Q_CLUSTER = { + 'name': 'DjangORM', + 'workers': 4, + 'timeout': 90, + 'retry': 120, + 'queue_limit': 50, + 'bulk': 10, + 'orm': 'default' + } + +Using the Django ORM backend will also enable the Queued Tasks table in the Admin. + +If you need better performance , you should consider using a different database backend than the main project. +Set ``orm`` to the name of that database connection and make sure you run migrations on it using the ``--database`` option. + .. _bulk: bulk @@ -256,6 +279,11 @@ cache For some brokers, you will need to set up the Django `cache framework `__ to gather statistics for the monitor. You can indicate which cache to use by setting this value. Defaults to ``default``. +scheduler +~~~~~~~~~ +You can disable the scheduler by setting this option to ``False``. This will reduce a little overhead if you're not using schedules, but is most useful if you want to temporarily disable all schedules. +Defaults to ``True`` + cpu_affinity ~~~~~~~~~~~~ diff --git a/docs/index.rst b/docs/index.rst index 4fc8886..ced20cc 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -20,7 +20,7 @@ Features - Django Admin integration - PaaS compatible with multiple instances - Multi cluster monitor -- Redis, Disque, IronMQ or SQS +- Redis, Disque, IronMQ, SQS or ORM - Python 2 and 3