Merge pull request #75 from Koed00/dev

Adds MongoDB broker

docs/brokers.rst

@@ -2,7 +2,7 @@ Brokers
 =======
 
 The broker sits between your Django instances and your Django Q cluster instances; accepting, saving and delivering task packages.
-Currently we support a variety of brokers from the default Redis, bleeding edge Disque to the convenient Amazon SQS.
+Currently we support a variety of brokers from the default Redis, bleeding edge Disque to the convenient ORM and fast MongoBD.
 
 The default Redis broker does not support message receipts.
 This means that in case of a catastrophic failure of the cluster server or worker timeouts, tasks that were being executed get lost.
@@ -73,6 +73,18 @@ Although `SQS <https://aws.amazon.com/sqs/>`__ is not the fastest, it is stable,
 * Requires the `boto3 <https://github.com/boto/boto3>`__ client library: ``pip install boto3``
 * See the :ref:`sqs_configuration` configuration section for options.
 
+
+MongoDB
+-------
+This highly scalable NoSQL database makes for a very fast and reliably persistent at-least-once message broker.
+Usually available on most PaaS providers, as `open-source <https://www.mongodb.org/>`__ or commercial `enterprise <https://www.mongodb.com/lp/download/mongodb-enterprise>`__ edition.
+
+* Delivery receipts
+* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
+* Can be configured as the Django cache-backend through several open-source cache providers.
+* Requires the `pymongo <https://github.com/mongodb/mongo-python-driver>`__ driver: ``pip install pymongo``
+* See the :ref:`mongo_configuration` configuration section for options.
+
 .. _orm_broker:
 
 Django ORM
@@ -84,6 +96,7 @@ However for a medium message rate and scheduled tasks, this is the most convenie
 * Delivery receipts
 * Supports bulk dequeue
 * Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
+* Can be `configured <https://docs.djangoproject.com/en/1.8/topics/cache/#database-caching>`__ as its own cache backend.
 * Queue editable in Django Admin
 * See the :ref:`orm_configuration` configuration on how to set it up.
 

docs/configure.rst

@@ -264,6 +264,34 @@ Using the Django ORM backend will also enable the Queued Tasks table in the Admi
 If you need better performance , you should consider using a different database backend than the main project.
 Set ``orm`` to the name of that database connection and make sure you run migrations on it using the ``--database`` option.
 
+.. _mongo_configuration:
+
+mongo
+~~~~~
+To use MongoDB as a message broker you simply provide the connection information in a dictionary::
+
+    # example MongoDB broker connection
+
+    Q_CLUSTER = {
+        'name': 'MongoDB',
+        'workers': 8,
+        'timeout': 60,
+        'retry': 70,
+        'queue_limit': 100,
+        'mongo': {
+                'host': '127.0.0.1',
+                'port': 27017
+        }
+    }
+
+The ``mongo`` dictionary can contain any of the parameters exposed by pymongo's `MongoClient <https://api.mongodb.org/python/current/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`__
+If you want to use a mongodb uri, you can supply it as the ``host`` parameter.
+
+mongo_db
+~~~~~~~~
+When using the MongoDB broker you can optionally provide a database name to use for the queues.
+Defaults to default database if available, otherwise ``django-q``
+
 .. _bulk:
 
 bulk
@@ -273,7 +301,7 @@ Especially HTTP based or very high latency servers can benefit from bulk dequeue
 Keep in mind however that settings this too high can degrade performance with multiple clusters or very large task packages.
 
 Not supported by the default Redis broker.
-Defaults to 1.
+Defaults to ``1``.
 
 cache
 ~~~~~

docs/index.rst

@@ -20,7 +20,7 @@ Features
 -  Django Admin integration
 -  PaaS compatible with multiple instances
 -  Multi cluster monitor
--  Redis, Disque, IronMQ, SQS or ORM
+-  Redis, Disque, IronMQ, SQS, MongoDB or ORM
 -  Python 2 and 3
 
 

docs/install.rst

@@ -68,10 +68,15 @@ Optional
 
     $ pip install iron-mq
 
+- `Pymongo <https://github.com/mongodb/mongo-python-driver>`__ is needed if you want to use MongoDB as a message broker::
+
+    $ pip install pymongo
+
 - `Redis <http://redis.io/>`__ server is the default broker for Django Q. It provides the best performance and does not require Django's cache framework for monitoring.
 
 - `Disque <https://github.com/antirez/disque>`__ server is based on Redis by the same author, but focuses on reliable queues. Currently in Alpha, but highly recommended. You can either build it from source or use it on Heroku through the `Tynd <https://disque.tynd.co/>`__ beta.
 
+- `MongoDB <https://www.mongodb.org/>`__ is a highly scalable NoSQL database which makes for a very fast and reliably persistent at-least-once message broker. Usually available on most PaaS providers.
 
 Compatibility
 -------------

docs/tasks.rst

@@ -79,6 +79,10 @@ a single keyword dict named ``q_options``. This enables you to use these keyword
 
 Please not that this will override any other option keywords.
 
+.. note::
+    For tasks to be processed you will need to have a worker cluster running in the background using ``python manage.py qcluster``
+    or you need to configure Django Q to run in synchronous mode for testing using the :ref:`sync` option.
+
 .. _groups:
 
 Groups