Commit Graph
326 Commits
Author SHA1 Message Date
Ilan Steemers 81d5abbd00 adds count and wait option to group result and fetch
result_group and fetch_group can now be told to block for a certain number of results or a number of miliseconds or a combination of both.
2015-10-02 18:29:03 +02:00
Ilan Steemers 9cfc41f1cb fixed global cached for fetch group
And added docstrings
2015-10-02 16:40:31 +02:00
Ilan Steemers f6d14bc7b3 adds delete_cached 2015-10-01 19:25:43 +02:00
Ilan Steemers 6127b551ae fixes tests for cache with better coverage 2015-10-01 17:21:26 +02:00
Ilan Steemers 7107e9ea35 fetch group should default to non cached 2015-10-01 15:06:43 +02:00
Ilan Steemers 33282cb2e5 cached result backend
first version of a result backend using django's cache framework
2015-10-01 14:58:28 +02:00
Ilan Steemers 07a7851b54 v0.7.7 2015-09-29 22:59:40 +02:00
Ilan Steemers 76aab3d52a v0.7.6 2015-09-29 18:31:56 +02:00
Ilan Steemers 00236990af Merge pull request #80 from Koed00/dev
#79  close django db connection before fork
2015-09-29 11:20:59 +02:00
Ilan Steemers 1aed7c2d00 #79 close django db connection before fork
The sentinel saves a statistic to the cache provider just before forking the workers. If this cache provider is a database backend, the connection stays open and gets forked and it crashes. Solution is to close any db connection before forking.
2015-09-29 10:47:22 +02:00
Ilan Steemers ca79228b95 v0.7.5 2015-09-28 18:23:02 +02:00
Ilan Steemers 765c84301f #76 checks for stale connections always
It turns out that checking stale connections on a timer takes between 1-2 times as long as just checking them always. This also has the benefit of catching timeouts that happen between timer loops.
2015-09-28 12:48:11 +02:00
Ilan Steemers 7075a91d37 #76 adds connection checked with timeout
Adds a check for old connections in both the workers and the monitor , every DB_TIMEOUT seconds
2015-09-28 11:42:48 +02:00
Ilan Steemers 4ceee42be9 #76 resets stale db connection on every scheduler ping 2015-09-28 10:42:08 +02:00
Ilan Steemers 08104ef777 check for default database on mongo connection
correction
2015-09-26 13:13:25 +02:00
Ilan Steemers b8bb186471 check for default database on mongo connection 2015-09-26 13:01:15 +02:00
Ilan Steemers 8bdbcdaf55 revert to info method 2015-09-25 21:27:33 +02:00
Ilan Steemers 826178c463 Adds a MongoDB broker
* adds mongo broker tests
* adds mongo brokers docs

* broker info is now cached when it needs to connect to a server, to reduce traffic.
2015-09-25 20:03:39 +02:00
Ilan Steemers ef6a60478a Merge pull request #74 from Koed00/dev
Removes root imports for Django 1.9
2015-09-25 12:35:10 +02:00
Ilan Steemers b3381d6dae removes root imports for django 1.9
typo in test
2015-09-24 15:47:40 +02:00
Ilan Steemers cef48da789 removes root imports for django 1.9
Updated docs to promote submodule imports
2015-09-24 15:41:32 +02:00
Ilan Steemers 131a37daf3 Removes Pycharm stdin bug workaround 2015-09-24 11:21:50 +02:00
Ilan Steemers 985fbb31cc tests cluster stop when already stopped 2015-09-24 11:21:28 +02:00
Ilan Steemers dc47ebef3c test for lock size in monitor 2015-09-19 17:25:02 +02:00
Ilan Steemers 7532cac93a test for lock_size in main class 2015-09-19 15:52:28 +02:00
Ilan Steemers 93a8f71157 Only show lock count when available and greater than zero 2015-09-19 15:42:35 +02:00
Ilan Steemers 370ffd4061 docs: Replaced redis pooling with broker pooling example.
* moves get_broker to main module
2015-09-18 21:07:18 +02:00
Ilan Steemers 270d8dba88 v0.7.3 2015-09-18 20:02:19 +02:00
Ilan Steemers 16c28a88e2 adds (approximate) lock_size to SQS broker
docs: added lock_size
2015-09-18 15:40:30 +02:00
Ilan Steemers 6af852fcee checks if broker has lock_size method 2015-09-18 15:23:31 +02:00
Ilan Steemers a493f9cc78 Adds wait option to result() and fetch() 2015-09-18 10:57:09 +02:00
Ilan Steemers a30e5d2c28 v0.7.2 2015-09-17 20:19:20 +02:00
Ilan Steemers 0d335d2254 Merge pull request #67 from Koed00/dev
orm: Improves locking behavior
2015-09-17 19:07:29 +02:00
Ilan Steemers a6ad47b181 Adds lock count to monitor for orm broker 2015-09-17 18:55:29 +02:00
Ilan Steemers 1d12107524 Changed locking mechanism
* all new tasks start as timeouts. This makes the queries a lot simpler.

* In BULK mode, the bulk update for the lock was causing actual database locks.  The lock updates are now iterated.
2015-09-17 18:49:03 +02:00
Ilan Steemers 8ec07cd478 Merge pull request #66 from Koed00/dev
tests orm queue admin overrides
2015-09-17 17:08:48 +02:00
Ilan Steemers 3ecd47fc7b tests orm queue admin overrides 2015-09-17 16:54:41 +02:00
Ilan Steemers ef9079a256 Merge pull request #65 from Koed00/dev
Adds remote orm admin view
2015-09-17 16:13:33 +02:00
Ilan Steemers 1ec1598200 Adds remote orm queue to admin view 2015-09-17 14:50:19 +02:00
Ilan Steemers 4d7416914c v0.7.1 2015-09-16 18:50:33 +02:00
Ilan Steemers 061775b3bb Adds group commands to task instances
* adds task.group_result()
* adds task.group_count()
* adds task.group_delete()
2015-09-16 15:47:39 +02:00
Ilan Steemers 1222285107 Adds --config option to qinfo command
Outputs current configuration constants
2015-09-15 21:39:38 +02:00
Ilan Steemers 548af6c4ad Tests no cache configuration 2015-09-15 20:29:09 +02:00
Ilan Steemers 7fb1142e4e Shares Sentinel broker with Monitor 2015-09-15 20:07:33 +02:00
Ilan Steemers dbf0b20ba5 Fixes test for admin view 2015-09-15 17:14:08 +02:00
Ilan Steemers 5450e59119 Adds name, function and task id to orm queue admin 2015-09-15 16:49:22 +02:00
Ilan Steemers a060b3508a Moves bulk iteration to pusher 2015-09-15 14:45:06 +02:00
Ilan Steemers 9b7c76b9d3 Getting ready for v0.7.0 2015-09-14 17:10:46 +02:00
Ilan Steemers ff5531aa99 Merge branch 'master' into orm
# Conflicts:
#	docs/conf.py
2015-09-14 16:57:28 +02:00
Ilan Steemers cefc7e0dc1 Adds test for orm queued task admin 2015-09-14 16:06:18 +02:00