Commit Graph
39 Commits
Author SHA1 Message Date
Ilan b99e16f108 adds catch_up configuration option
When a cluster hasn't run for a while, the default behavior is to keep executing schedules until they are caught up with the present time.
By setting `catch_up` to False, schedules will not play catch up and instead get rescheduled to the future.
2015-08-18 11:57:15 +02:00
Ilan ac2dc02b4c consolidated redis ping 2015-08-17 14:25:32 +02:00
Ilan 47a92f8e56 added to translations
fixed _ conflict
2015-08-17 12:57:59 +02:00
Ilan de5d0f2725 Adds a 'minutes' option to the scheduler 2015-08-05 20:30:29 +02:00
Ilan f470fbf975 uses empty() instead of qsize() == 0 2015-08-04 18:30:07 +02:00
Ilan ae8745ab39 Checks if cpu_count is implemented
On rare occasions, usually on win32, cpu_count will raise a NotImplemented error.  If psutil is installed we will use this to get the correct count, otherwise we default to 4
2015-08-04 16:28:19 +02:00
Ilan af61270f62 #40 qsize not implemented 2015-08-04 12:26:23 +02:00
Ilan 9899c42740 * fixes save_limit +1
* adds save_limit test
* expands recycle test
2015-07-31 18:39:54 +02:00
Ilan Steemers f75593aebb Moves unpacking task from Worker to Pusher.
Makes more sense and gives us more options to work with control queues from the pusher in the future. Does not seem to affect performance.
2015-07-31 12:29:10 +02:00
Ilan ba58b4face closes db connection on worker and monitor spawn
This should prevent some problems with Postgresql where the workers would use stale db connections  and cause errors.
2015-07-27 12:31:38 +02:00
Ilan cf325f8199 queue block is true by default. 2015-07-24 10:52:30 +02:00
Ilan d31e9e2b97 adds a limit to the queue 2015-07-24 01:04:58 +02:00
Ilan a8b562bc04 some minor linting 2015-07-22 17:13:29 +02:00
Ilan Steemers a26ffbf03b tuning timeout 2015-07-22 14:25:32 +02:00
Ilan f6cbd9817a fix for timeout 2015-07-22 13:11:52 +02:00
Ilan c5b8b800f6 Adds save override option for tasks
Also adds `q_options` dict for async and schedule options
2015-07-20 19:43:27 +02:00
Ilan c127604f10 Adds task groups
* task now takes an optional keyword `group`
* added result_group()  and fetch_group()
* added group column to Success admin
* group column in searchable
* added `name` field to Schedules
* Scheduled tasks take schedule name as group
2015-07-18 17:20:50 +02:00
Ilan b876b1e06d Tests now run with logging level DEBUG 2015-07-17 14:17:40 +02:00
Ilan af5e99249f Paste accidents happen 2015-07-16 20:47:14 +02:00
Ilan af735c328e Adds a timeout override per task
* added a `timeout` keyword to tasks.
* timeouts now count down instead of up
* `timeout` in task overrides global timeout setting
* added tests for timeout override
* added a test for recycling
2015-07-16 20:35:29 +02:00
Ilan 1e81cc413d Missing newline 2015-07-15 22:50:53 +02:00
Ilan a5580c75a1 Workaround for Travis
Travis doesn't support cpu affinity in their virtuals so I've added a setting TESTING that will bypass the actual affinity setting, but will still cover most of the code.
Might be useful for future workarounds.
2015-07-15 22:17:13 +02:00
Ilan 2d0820e80b Adds cpu affinity for the workers. Courtesy of the optional psutil module. 2015-07-15 19:53:13 +02:00
Ilan Steemers 6c9dd1a910 Schedules of type ONCE will now self destruct with negative repeats or just deactivated with positive repeats 2015-07-11 11:32:46 +02:00
Ilan Steemers e590628612 linting some blank lines 2015-07-10 12:14:57 +02:00
Ilan Steemers 90d30e14e8 fixed regression of the save pruning due to introduction of uuid as primary key. 2015-07-10 12:07:27 +02:00
Ilan Steemers e62fd1ee70 Fixes #7
* added a synchronous cluster simulation

* added a test for the sync function

* moved signing into its own module

* refactored imports to solve circular import problems stemming from the task module pretending to be a cluster

* added documentation for the new sync option
2015-07-09 20:16:21 +02:00
Ilan Steemers f8708cfd73 Task now uses a UUID4 instead of a LUID 2015-07-08 13:33:03 +02:00
Ilan Steemers 729abaf726 set exit timeout if no timeout is given 2015-07-07 20:37:03 +02:00
Ilan Steemers b05fe76fbc log database errors more quiet 2015-07-07 17:54:24 +02:00
Ilan Steemers b801c480f5 Handle Redis errors better 2015-07-07 17:47:49 +02:00
Ilan Steemers 33b7fe50ed fix reincarnation bug without timeout set 2015-07-07 16:45:05 +02:00
Ilan Steemers 38dc20043c Stabilize the stop procedure after forced termination of the monitor causes an unstable queue.
* added timeout to final stop
* ony terminate on timeouts
* renamed done_queue to result_queue
2015-07-07 15:48:13 +02:00
Ilan Steemers 254d99e63b Fix reincarnation bug 2015-07-06 20:54:24 +02:00
Ilan Steemers d06aa40a38 Removed Cluster.is_idle
It wasn't used and it could be confused with the cluster stat IDLE
2015-07-06 11:13:54 +02:00
Ilan Steemers 70ea211cfa Can spot difference between recycle and death now.
Added recycle flag
Changed pids to names for readability
2015-07-06 10:42:38 +02:00
Ilan Steemers 83b12629ea removed trailing period 2015-07-03 19:57:22 +02:00
Ilan Steemers 29774b3afe raises a redis exception 2015-07-03 18:30:22 +02:00
Ilan Steemers b113a52825 Restructured all the modules
The core module was becoming too large.
By splitting it in  cluster, monitor and tasks it becomes more manageable. It also reflects the documentation structure.
2015-07-03 15:12:11 +02:00