mirror of
https://github.com/django-q2/django-q2.git
synced 2026-09-29 20:58:12 +08:00
Adds a MongoDB broker
* adds mongo broker tests * adds mongo brokers docs * broker info is now cached when it needs to connect to a server, to reduce traffic.
This commit is contained in:
+11
-1
@@ -2,7 +2,7 @@ Brokers
|
||||
=======
|
||||
|
||||
The broker sits between your Django instances and your Django Q cluster instances; accepting, saving and delivering task packages.
|
||||
Currently we support a variety of brokers from the default Redis, bleeding edge Disque to the convenient Amazon SQS.
|
||||
Currently we support a variety of brokers from the default Redis, bleeding edge Disque to the convenient ORM and fast MongoBD.
|
||||
|
||||
The default Redis broker does not support message receipts.
|
||||
This means that in case of a catastrophic failure of the cluster server or worker timeouts, tasks that were being executed get lost.
|
||||
@@ -73,6 +73,16 @@ Although `SQS <https://aws.amazon.com/sqs/>`__ is not the fastest, it is stable,
|
||||
* Requires the `boto3 <https://github.com/boto/boto3>`__ client library: ``pip install boto3``
|
||||
* See the :ref:`sqs_configuration` configuration section for options.
|
||||
|
||||
|
||||
MongoDB
|
||||
-------
|
||||
The
|
||||
|
||||
* Delivery receipts
|
||||
* Needs Django's `Cache framework <https://docs.djangoproject.com/en/1.8/topics/cache/#setting-up-the-cache>`__ configured for monitoring
|
||||
* Requires the `pymongo <https://github.com/mongodb/mongo-python-driver>`__ driver: ``pip install pymongo``
|
||||
* See the :ref:`mongo_configuration` configuration section for options.
|
||||
|
||||
.. _orm_broker:
|
||||
|
||||
Django ORM
|
||||
|
||||
+28
-1
@@ -264,6 +264,33 @@ Using the Django ORM backend will also enable the Queued Tasks table in the Admi
|
||||
If you need better performance , you should consider using a different database backend than the main project.
|
||||
Set ``orm`` to the name of that database connection and make sure you run migrations on it using the ``--database`` option.
|
||||
|
||||
.. _mongo_configuration:
|
||||
|
||||
mongo
|
||||
~~~~~
|
||||
To use MongoDB as a message broker you simply provide the connection information in a dictionary ::
|
||||
|
||||
# example MongoDB broker connection
|
||||
|
||||
Q_CLUSTER = {
|
||||
'name': 'MongoDB',
|
||||
'workers': 8,
|
||||
'timeout': 60,
|
||||
'retry': 70,
|
||||
'queue_limit': 100,
|
||||
'mongo': {
|
||||
'host': '127.0.0.1',
|
||||
'port': 27017
|
||||
}
|
||||
}
|
||||
|
||||
The ``mongo`` dictionary can contain any of the parameters exposed by pymongo's`MongoClient<https://api.mongodb.org/python/current/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`__
|
||||
|
||||
mongo_db
|
||||
~~~~~~~~
|
||||
When using the MongoDB broker you can optionally provide a database name to use for the queues.
|
||||
Defaults to ``django-q``
|
||||
|
||||
.. _bulk:
|
||||
|
||||
bulk
|
||||
@@ -273,7 +300,7 @@ Especially HTTP based or very high latency servers can benefit from bulk dequeue
|
||||
Keep in mind however that settings this too high can degrade performance with multiple clusters or very large task packages.
|
||||
|
||||
Not supported by the default Redis broker.
|
||||
Defaults to 1.
|
||||
Defaults to ``1``.
|
||||
|
||||
cache
|
||||
~~~~~
|
||||
|
||||
Reference in New Issue
Block a user