According to multiprocessing documentation for Value
(https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Value)
reads and writes are protected with lock when the lock argument is True
(the default) or the lock argument is an instance of Lock or RLock. The
documentation states that operations like += are not atomic as that
involves reading and writing.
On the worker side the critical section includes also storing finished task
result because the timeout could happen after the task function has
finished but before the result has been stored and timer.value has been
updated to tell the guard process that the task has been finished.
On the guard side the critical section includes all checks done to see
if the worker has timed out or died and the actual reincarnation function
because the worker could update timer value to -1 (idle) or -2 (recycle)
after the guard has seen timer value 0 (timeout) and is going to terminate
the worker.
The cluster timeout configuration default value None is documented to mean
tasks never timeout out. Also the documentation states that the timeout
can be overridden for individual tasks.
With the old implementation the timeout parameter given to async_task was
not honored if the cluster timeout was set to None.
Fixes: #335
* async is now a reserved word in python3.7
* Rename async function to enqueue
* Rename all async_ functions to enqueue_
* Rename Async class to AsyncTask
* Updates the docs.
If a task fails with an exception, it is retried until it
succeeds. This is contrary to what is said in the documentation: under
the "Architecture" section, heading "Broker" it says that even when a
task errors, it's still considered a successful delivery. Failed tasks
never get acknowledged however, thereby being retried after the
timeout period. See also issues #238 and #194.
This patch adds an option to acknowledge failures, thereby closing
issue #238. Issue #194 would require some more work. The default of
this option is set to `False`, thereby maintaining backwards
compatibility.
With async iter you can quickly run the same function on an iterable set of arguments. The results are held in the cache until all are done and collated into a database result.
The sentinel saves a statistic to the cache provider just before forking the workers. If this cache provider is a database backend, the connection stays open and gets forked and it crashes. Solution is to close any db connection before forking.
It turns out that checking stale connections on a timer takes between 1-2 times as long as just checking them always. This also has the benefit of catching timeouts that happen between timer loops.
Ilan Steemers