Ilan Steemers
3cc4021aad
Optimizes option iteration for async
...
Simplified option iteration to only use a single loop. This makes the async command about twice as fast in optimal conditions.
2015-10-27 10:29:35 +01:00
Ilan Steemers
12bfd47917
Splits options from keywords through iteration
...
Popping options from async kwargs will make any reference to them void. Which can lead to unwanted situations. Unfortunately deepcopy clashes with the multiprocessing so we resort to simple re-iteration.
2015-10-26 15:33:42 +01:00
Ilan Steemers
728ba45ef0
Adds Async class wrapper for the async function
2015-10-26 13:48:23 +01:00
Ilan Steemers
62337a9d06
Adds infinite wait to task result functions
...
* the `wait` keyword now can take the value `-1` which will make the functions wait indefinitely for a result.
2015-10-25 16:40:49 +01:00
Ilan Steemers
7eefb8cb64
returns the size of the chain or iter on append
2015-10-18 15:57:23 +02:00
Ilan Steemers
eb72eb33d9
Adds an Iter class
...
The Iter class serves as a convenience rwrapper around the `async_iter` function
2015-10-18 13:53:32 +02:00
Ilan Steemers
083715f9bd
Replaces list.copy with a slice for python 2.7
2015-10-17 18:59:17 +02:00
Ilan Steemers
2df29b908c
Adds tests for task chains
...
adds several small improvements based on the problems that surfaced during writing the tests
2015-10-17 18:49:45 +02:00
Ilan Steemers
6dcead7310
Adds task chains
2015-10-17 12:13:29 +02:00
Ilan Steemers
2ba88f2d17
adds cached option to async_iter
2015-10-06 19:37:02 +02:00
Ilan Steemers
f73ae66c0f
Adds async_iter command
...
With async iter you can quickly run the same function on an iterable set of arguments. The results are held in the cache until all are done and collated into a database result.
2015-10-03 18:07:57 +02:00
Ilan Steemers
81d5abbd00
adds count and wait option to group result and fetch
...
result_group and fetch_group can now be told to block for a certain number of results or a number of miliseconds or a combination of both.
2015-10-02 18:29:03 +02:00
Ilan Steemers
9cfc41f1cb
fixed global cached for fetch group
...
And added docstrings
2015-10-02 16:40:31 +02:00
Ilan Steemers
f6d14bc7b3
adds delete_cached
2015-10-01 19:25:43 +02:00
Ilan Steemers
6127b551ae
fixes tests for cache with better coverage
2015-10-01 17:21:26 +02:00
Ilan Steemers
7107e9ea35
fetch group should default to non cached
2015-10-01 15:06:43 +02:00
Ilan Steemers
33282cb2e5
cached result backend
...
first version of a result backend using django's cache framework
2015-10-01 14:58:28 +02:00
Ilan Steemers
a493f9cc78
Adds wait option to result() and fetch()
2015-09-18 10:57:09 +02:00
Ilan Steemers
da20ebe734
fixes connection creation on import
2015-08-31 15:15:24 +02:00
Ilan Steemers
0740c90148
adds pluggable brokers
...
* added redis broker
* added django_redis broker
* added task acknowledgement
2015-08-29 18:54:07 +02:00
Ilan
6c37b09151
adds global sync configuration option
...
Setting the `sync` configuration option will force all async() calls to be run with `sync=True`. Useful for testing.
2015-08-13 18:02:38 +02:00
Ilan
3d12c1d36e
adds queue_size()
...
returns the size of the current redis queue
2015-08-13 15:54:57 +02:00
Ilan
de5d0f2725
Adds a 'minutes' option to the scheduler
2015-08-05 20:30:29 +02:00
Ilan Steemers
f75593aebb
Moves unpacking task from Worker to Pusher.
...
Makes more sense and gives us more options to work with control queues from the pusher in the future. Does not seem to affect performance.
2015-07-31 12:29:10 +02:00
Ilan
266a0dbcb1
minor linting
2015-07-24 13:36:51 +02:00
Ilan
1a24bb60dd
removed old import
2015-07-22 17:03:49 +02:00
Ilan
c5b8b800f6
Adds save override option for tasks
...
Also adds `q_options` dict for async and schedule options
2015-07-20 19:43:27 +02:00
Ilan
52e72fde1c
Adds failed tasks filtering to groups
2015-07-19 15:15:57 +02:00
Ilan
ff102630f7
adds count_group() and delete_group()
2015-07-19 13:19:46 +02:00
Ilan
e4737d765a
Schedule name is optional
...
For backwards compatibility the schedule name is an optional keyword in the `schedule` function.
Adjusted docs accordingly
2015-07-18 19:41:21 +02:00
Ilan
a5d4c5186c
renamed get_group_result to get_result_group
2015-07-18 18:49:50 +02:00
Ilan
c127604f10
Adds task groups
...
* task now takes an optional keyword `group`
* added result_group() and fetch_group()
* added group column to Success admin
* group column in searchable
* added `name` field to Schedules
* Scheduled tasks take schedule name as group
2015-07-18 17:20:50 +02:00
Ilan Steemers
e590628612
linting some blank lines
2015-07-10 12:14:57 +02:00
Ilan Steemers
e62fd1ee70
Fixes #7
...
* added a synchronous cluster simulation
* added a test for the sync function
* moved signing into its own module
* refactored imports to solve circular import problems stemming from the task module pretending to be a cluster
* added documentation for the new sync option
2015-07-09 20:16:21 +02:00
Ilan Steemers
f8708cfd73
Task now uses a UUID4 instead of a LUID
2015-07-08 13:33:03 +02:00
Ilan Steemers
b113a52825
Restructured all the modules
...
The core module was becoming too large.
By splitting it in cluster, monitor and tasks it becomes more manageable. It also reflects the documentation structure.
2015-07-03 15:12:11 +02:00
Ilan Steemers