mirror of
https://github.com/django-q2/django-q2.git
synced 2026-09-23 23:08:11 +08:00
Updated architecture docs
This commit is contained in:
+13
-15
@@ -101,17 +101,15 @@ Pusher
|
||||
""""""
|
||||
|
||||
The pusher process continuously checks the Redis list for new task
|
||||
packages and pushes them on the Task Queue.
|
||||
packages. It checks the signing and unpacks the task to the Task Queue.
|
||||
|
||||
Worker
|
||||
""""""
|
||||
|
||||
A worker process pulls a package of the Task Queue and checks the signing and unpacks the task.
|
||||
Before executing the task it set a timer on the :ref:`sentinel` indicating its about to start work.
|
||||
Afterwards it the timer is reset and any results (including errors) are saved to the package.
|
||||
A worker process pulls a task of the Task Queue and it sets a shared countdown timer with :ref:`sentinel` indicating it is about to start work.
|
||||
The worker then tries to execute the task and afterwards the timer is reset and any results (including errors) are saved to the package.
|
||||
Irrespective of the failure or success of any of these steps, the package is then pushed onto the Result Queue.
|
||||
|
||||
|
||||
Monitor
|
||||
"""""""
|
||||
|
||||
@@ -125,38 +123,38 @@ Sentinel
|
||||
|
||||
The sentinel spawns all process and then checks the health of all
|
||||
workers, including the pusher and the monitor. This includes checking timers on each worker for timeouts.
|
||||
In case of a sudden death or timeout, it will reincarnate the failing processes. When a stop signal, the sentinel will halt the
|
||||
In case of a sudden death or timeout, it will reincarnate the failing processes. When a stop signal is received, the sentinel will halt the
|
||||
pusher and instruct the workers and monitor to finish the remaining items. See :ref:`stop_procedure`
|
||||
|
||||
Timeouts
|
||||
""""""""
|
||||
Before each task execution the worker resets a timer on the sentinel and resets it again after execution.
|
||||
Meanwhile the the sentinel checks if the timers don't exceed the timeout amount, in which case it will terminate the worker and reincarnate a new one.
|
||||
Before each task execution the worker sets a countdown timer on the sentinel and resets it again after execution.
|
||||
Meanwhile the sentinel checks if the timers don't reach zero, in which case it will terminate the worker and reincarnate a new one.
|
||||
|
||||
Scheduler
|
||||
"""""""""
|
||||
Once a minute the scheduler checks for any scheduled tasks that should be starting.
|
||||
Twice a minute the scheduler checks for any scheduled tasks that should be starting.
|
||||
|
||||
- Creates a task from the schedule
|
||||
- Subtracts 1 from :attr:`django_q.Schedule.repeats`
|
||||
- Sets the next run time if there are repeats left or if its negative.
|
||||
- Sets the next run time if there are repeats left or if it has a negative value.
|
||||
|
||||
.. _stop_procedure:
|
||||
|
||||
Stop procedure
|
||||
""""""""""""""
|
||||
|
||||
When a stop signal is given, the sentinel exits the guard loop and instructs the pusher to stop pushing.
|
||||
Once this is confirmed, the sentinel pushes poison pills onto the task queue and will wait for all the workers to die.
|
||||
This ensures that the queue is emptied before the workers exit.
|
||||
Afterwards the sentinel waits for the monitor to empty the result and then the stop procedure is complete.
|
||||
When a stop signal is received, the sentinel exits the guard loop and instructs the pusher to stop pushing.
|
||||
Once this is confirmed, the sentinel pushes poison pills onto the task queue and will wait for all the workers to exit.
|
||||
This ensures that the task queue is emptied before the workers exit.
|
||||
Afterwards the sentinel waits for the monitor to empty the result queue and the stop procedure is complete.
|
||||
|
||||
- Send stop event to pusher
|
||||
- Wait for pusher to exit
|
||||
- Put poison pills in the Task Queue
|
||||
- Wait for all the workers to clear the queue and stop
|
||||
- Put a poison pill on the Result Queue
|
||||
- Wait for monitor to process remaining results
|
||||
- Wait for monitor to process remaining results and exit
|
||||
- Signal that we have stopped
|
||||
|
||||
.. warning::
|
||||
|
||||
Reference in New Issue
Block a user