Multiple queue, multiple cluster in one site (#71)

* Support for multi-queue, multi-cluster configuration. API changes include:
  * Adding `cluster` to async_task() parameters and Task model
  * Adding argument --name to qcluster command
  * Necessary adjustments to Conf and Broker classes
  * Some admin improvements

* Add settings.Q_CLUSTER['ALT_CLUSTERS']: q_cluster config overrides for alternative clusters;
Add Conf.CLUSTER_NAME: separate usage from Conf.PREFIX;
QueueAdmin/OrmQ detail page enhanced: now displaying args/kwargs/q_options instead of encrypted payload.

* if `cluster` argument is not set (the default), async_task() and schedule() will be handled by the default cluster; Documentation update.

* Text cleanup

* Documentation update.

* Fix TIMEOUT setting in Windows for non-default cluster

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>

docs/cluster.rst

@@ -52,6 +52,30 @@ You can have multiple clusters on multiple machines, working on the same queue a
 - They use the same cluster name. See :doc:`configure`
 - They share the same ``SECRET_KEY`` for Django.
 
+.. _multiple-queues
+
+Multiple Queues
+-----------------
+You can have multiple queues in one Django site, and use multiple cluster to work on each queue.
+Different queues are identified by different queue names which are also cluster names.
+To run an alternate cluster, e.g. to work on the 'long' queue, start your cluster with command::
+
+    # On Linux
+    $ Q_CLUSTER_NAME=long python manage.py qcluster
+
+    # On Windows
+    $ python manage.py qcluster --name long
+
+You can set different Q_CLUSTER options for alternative clusters, such as 'timeout', 'queue_limit'
+and any other options which are valid in :doc:`configure`. See :ref:`alt-clusters`.
+
+.. note::
+
+    To use multiple queue, use the keyword argument `cluster` in async_task() and schedule():
+
+    * if `cluster` is not set (the default), async_task() and schedule() will be handled by the default cluster;
+    * if `cluster` is set, only clusters with matching cluster name will run the task or do the schedule.
+
 Using a Procfile
 ----------------
 If you host on `Heroku <https://heroku.com>`__ or you are using `Honcho <https://github.com/nickstenning/honcho>`__ you can start the cluster from a :file:`Procfile` with an entry like this::

docs/configure.rst

@@ -20,7 +20,18 @@ Configuration is handled via the ``Q_CLUSTER`` dictionary in your :file:`setting
         'redis': {
             'host': '127.0.0.1',
             'port': 6379,
-            'db': 0, }
+            'db': 0, },
+        'ALT_CLUSTERS': {
+            'long': {
+                'timeout': 3000,
+                'retry': 3600,
+                'max_attempts': 2,
+            },
+            'short': {
+                'timeout': 10,
+                'max_attempts': 1,
+            },
+        }
     }
 
 All configuration settings are optional:
@@ -459,6 +470,24 @@ As a rule of thumb; cpu_affinity 1 favors repetitive short running tasks, while
 
     *Psutil does not support cpu affinity on OS X at this time.*
 
+.. _alt-clusters:
+
+ALT_CLUSTERS
+~~~~~~~~~~~~
+
+For multiple clusters working on multiple queues to run in one Django site.
+ALT_CLUSTERS should be a dict with cluster_name as its key, and the value is the configuration for the cluster
+with the key as its name. The configuration items are consistent with Q_CLUSTER,
+except for a few items such as name/cluster_name/ALT_CLUSTER, which are not available of course.
+
+See :ref:`multiple-queues`.
+
+.. note::
+
+    For a cluster, if its name is in ALT_CLUSTERS, the config item in ALT_CLUSTER will override
+    the same config item in the Q_CLUSTER root. Other config items in Q_CLUSTER root remain in effect for this cluster.
+
+
 .. py:module:: django_q
 
 .. rubric:: Footnotes