Ilan Steemers
1ccfc66254
adds fail method to broker
2015-09-03 16:34:00 +02:00
Ilan Steemers
da20ebe734
fixes connection creation on import
2015-08-31 15:15:24 +02:00
Ilan Steemers
0740c90148
adds pluggable brokers
...
* added redis broker
* added django_redis broker
* added task acknowledgement
2015-08-29 18:54:07 +02:00
Ilan Steemers
439c8a2f77
Moves Status to it's own file
...
This way only qmonitor and qinfo need Curses and you can still run the cluster without it.
2015-08-26 13:11:58 +02:00
Ilan Steemers
3e92fddcdd
#50 removes redis connection argument from pusher process
2015-08-24 10:08:43 +02:00
Ilan Steemers
b414b637eb
#50 adds psutil as alternative os.getppid provider
2015-08-23 21:19:35 +02:00
Ilan
b99e16f108
adds catch_up configuration option
...
When a cluster hasn't run for a while, the default behavior is to keep executing schedules until they are caught up with the present time.
By setting `catch_up` to False, schedules will not play catch up and instead get rescheduled to the future.
2015-08-18 11:57:15 +02:00
Ilan
ac2dc02b4c
consolidated redis ping
2015-08-17 14:25:32 +02:00
Ilan
47a92f8e56
added to translations
...
fixed _ conflict
2015-08-17 12:57:59 +02:00
Ilan
de5d0f2725
Adds a 'minutes' option to the scheduler
2015-08-05 20:30:29 +02:00
Ilan
f470fbf975
uses empty() instead of qsize() == 0
2015-08-04 18:30:07 +02:00
Ilan
ae8745ab39
Checks if cpu_count is implemented
...
On rare occasions, usually on win32, cpu_count will raise a NotImplemented error. If psutil is installed we will use this to get the correct count, otherwise we default to 4
2015-08-04 16:28:19 +02:00
Ilan
af61270f62
#40 qsize not implemented
2015-08-04 12:26:23 +02:00
Ilan
9899c42740
* fixes save_limit +1
...
* adds save_limit test
* expands recycle test
2015-07-31 18:39:54 +02:00
Ilan Steemers
f75593aebb
Moves unpacking task from Worker to Pusher.
...
Makes more sense and gives us more options to work with control queues from the pusher in the future. Does not seem to affect performance.
2015-07-31 12:29:10 +02:00
Ilan
ba58b4face
closes db connection on worker and monitor spawn
...
This should prevent some problems with Postgresql where the workers would use stale db connections and cause errors.
2015-07-27 12:31:38 +02:00
Ilan
cf325f8199
queue block is true by default.
2015-07-24 10:52:30 +02:00
Ilan
d31e9e2b97
adds a limit to the queue
2015-07-24 01:04:58 +02:00
Ilan
a8b562bc04
some minor linting
2015-07-22 17:13:29 +02:00
Ilan Steemers
a26ffbf03b
tuning timeout
2015-07-22 14:25:32 +02:00
Ilan
f6cbd9817a
fix for timeout
2015-07-22 13:11:52 +02:00
Ilan
c5b8b800f6
Adds save override option for tasks
...
Also adds `q_options` dict for async and schedule options
2015-07-20 19:43:27 +02:00
Ilan
c127604f10
Adds task groups
...
* task now takes an optional keyword `group`
* added result_group() and fetch_group()
* added group column to Success admin
* group column in searchable
* added `name` field to Schedules
* Scheduled tasks take schedule name as group
2015-07-18 17:20:50 +02:00
Ilan
b876b1e06d
Tests now run with logging level DEBUG
2015-07-17 14:17:40 +02:00
Ilan
af5e99249f
Paste accidents happen
2015-07-16 20:47:14 +02:00
Ilan
af735c328e
Adds a timeout override per task
...
* added a `timeout` keyword to tasks.
* timeouts now count down instead of up
* `timeout` in task overrides global timeout setting
* added tests for timeout override
* added a test for recycling
2015-07-16 20:35:29 +02:00
Ilan
1e81cc413d
Missing newline
2015-07-15 22:50:53 +02:00
Ilan
a5580c75a1
Workaround for Travis
...
Travis doesn't support cpu affinity in their virtuals so I've added a setting TESTING that will bypass the actual affinity setting, but will still cover most of the code.
Might be useful for future workarounds.
2015-07-15 22:17:13 +02:00
Ilan
2d0820e80b
Adds cpu affinity for the workers. Courtesy of the optional psutil module.
2015-07-15 19:53:13 +02:00
Ilan Steemers
6c9dd1a910
Schedules of type ONCE will now self destruct with negative repeats or just deactivated with positive repeats
2015-07-11 11:32:46 +02:00
Ilan Steemers
e590628612
linting some blank lines
2015-07-10 12:14:57 +02:00
Ilan Steemers
90d30e14e8
fixed regression of the save pruning due to introduction of uuid as primary key.
2015-07-10 12:07:27 +02:00
Ilan Steemers
e62fd1ee70
Fixes #7
...
* added a synchronous cluster simulation
* added a test for the sync function
* moved signing into its own module
* refactored imports to solve circular import problems stemming from the task module pretending to be a cluster
* added documentation for the new sync option
2015-07-09 20:16:21 +02:00
Ilan Steemers
f8708cfd73
Task now uses a UUID4 instead of a LUID
2015-07-08 13:33:03 +02:00
Ilan Steemers
729abaf726
set exit timeout if no timeout is given
2015-07-07 20:37:03 +02:00
Ilan Steemers
b05fe76fbc
log database errors more quiet
2015-07-07 17:54:24 +02:00
Ilan Steemers
b801c480f5
Handle Redis errors better
2015-07-07 17:47:49 +02:00
Ilan Steemers
33b7fe50ed
fix reincarnation bug without timeout set
2015-07-07 16:45:05 +02:00
Ilan Steemers
38dc20043c
Stabilize the stop procedure after forced termination of the monitor causes an unstable queue.
...
* added timeout to final stop
* ony terminate on timeouts
* renamed done_queue to result_queue
2015-07-07 15:48:13 +02:00
Ilan Steemers
254d99e63b
Fix reincarnation bug
2015-07-06 20:54:24 +02:00
Ilan Steemers
d06aa40a38
Removed Cluster.is_idle
...
It wasn't used and it could be confused with the cluster stat IDLE
2015-07-06 11:13:54 +02:00
Ilan Steemers
70ea211cfa
Can spot difference between recycle and death now.
...
Added recycle flag
Changed pids to names for readability
2015-07-06 10:42:38 +02:00
Ilan Steemers
83b12629ea
removed trailing period
2015-07-03 19:57:22 +02:00
Ilan Steemers
29774b3afe
raises a redis exception
2015-07-03 18:30:22 +02:00
Ilan Steemers
b113a52825
Restructured all the modules
...
The core module was becoming too large.
By splitting it in cluster, monitor and tasks it becomes more manageable. It also reflects the documentation structure.
2015-07-03 15:12:11 +02:00
Ilan Steemers