Updated architecture docs

This commit is contained in:
Ilan
2015-07-31 12:45:57 +02:00
parent f75593aebb
commit a1c67c819b
+13 -15
View File
@@ -101,17 +101,15 @@ Pusher
"""""" """"""
The pusher process continuously checks the Redis list for new task The pusher process continuously checks the Redis list for new task
packages and pushes them on the Task Queue. packages. It checks the signing and unpacks the task to the Task Queue.
Worker Worker
"""""" """"""
A worker process pulls a package of the Task Queue and checks the signing and unpacks the task. A worker process pulls a task of the Task Queue and it sets a shared countdown timer with :ref:`sentinel` indicating it is about to start work.
Before executing the task it set a timer on the :ref:`sentinel` indicating its about to start work. The worker then tries to execute the task and afterwards the timer is reset and any results (including errors) are saved to the package.
Afterwards it the timer is reset and any results (including errors) are saved to the package.
Irrespective of the failure or success of any of these steps, the package is then pushed onto the Result Queue. Irrespective of the failure or success of any of these steps, the package is then pushed onto the Result Queue.
Monitor Monitor
""""""" """""""
@@ -125,38 +123,38 @@ Sentinel
The sentinel spawns all process and then checks the health of all The sentinel spawns all process and then checks the health of all
workers, including the pusher and the monitor. This includes checking timers on each worker for timeouts. workers, including the pusher and the monitor. This includes checking timers on each worker for timeouts.
In case of a sudden death or timeout, it will reincarnate the failing processes. When a stop signal, the sentinel will halt the In case of a sudden death or timeout, it will reincarnate the failing processes. When a stop signal is received, the sentinel will halt the
pusher and instruct the workers and monitor to finish the remaining items. See :ref:`stop_procedure` pusher and instruct the workers and monitor to finish the remaining items. See :ref:`stop_procedure`
Timeouts Timeouts
"""""""" """"""""
Before each task execution the worker resets a timer on the sentinel and resets it again after execution. Before each task execution the worker sets a countdown timer on the sentinel and resets it again after execution.
Meanwhile the the sentinel checks if the timers don't exceed the timeout amount, in which case it will terminate the worker and reincarnate a new one. Meanwhile the sentinel checks if the timers don't reach zero, in which case it will terminate the worker and reincarnate a new one.
Scheduler Scheduler
""""""""" """""""""
Once a minute the scheduler checks for any scheduled tasks that should be starting. Twice a minute the scheduler checks for any scheduled tasks that should be starting.
- Creates a task from the schedule - Creates a task from the schedule
- Subtracts 1 from :attr:`django_q.Schedule.repeats` - Subtracts 1 from :attr:`django_q.Schedule.repeats`
- Sets the next run time if there are repeats left or if its negative. - Sets the next run time if there are repeats left or if it has a negative value.
.. _stop_procedure: .. _stop_procedure:
Stop procedure Stop procedure
"""""""""""""" """"""""""""""
When a stop signal is given, the sentinel exits the guard loop and instructs the pusher to stop pushing. When a stop signal is received, the sentinel exits the guard loop and instructs the pusher to stop pushing.
Once this is confirmed, the sentinel pushes poison pills onto the task queue and will wait for all the workers to die. Once this is confirmed, the sentinel pushes poison pills onto the task queue and will wait for all the workers to exit.
This ensures that the queue is emptied before the workers exit. This ensures that the task queue is emptied before the workers exit.
Afterwards the sentinel waits for the monitor to empty the result and then the stop procedure is complete. Afterwards the sentinel waits for the monitor to empty the result queue and the stop procedure is complete.
- Send stop event to pusher - Send stop event to pusher
- Wait for pusher to exit - Wait for pusher to exit
- Put poison pills in the Task Queue - Put poison pills in the Task Queue
- Wait for all the workers to clear the queue and stop - Wait for all the workers to clear the queue and stop
- Put a poison pill on the Result Queue - Put a poison pill on the Result Queue
- Wait for monitor to process remaining results - Wait for monitor to process remaining results and exit
- Signal that we have stopped - Signal that we have stopped
.. warning:: .. warning::