Commit Graph

164 Commits

Author SHA1 Message Date
David Hoover
4ba9e0338a Add post_spawn signal. (#93)
Co-authored-by: David Hoover <hooverd@arizona.edu>
2023-05-14 02:51:48 +02:00
sinowood
52c04217e9 Multiple queue, multiple cluster in one site (#71)
* Support for multi-queue, multi-cluster configuration. API changes include:
  * Adding `cluster` to async_task() parameters and Task model
  * Adding argument --name to qcluster command
  * Necessary adjustments to Conf and Broker classes
  * Some admin improvements

* Add settings.Q_CLUSTER['ALT_CLUSTERS']: q_cluster config overrides for alternative clusters;
Add Conf.CLUSTER_NAME: separate usage from Conf.PREFIX;
QueueAdmin/OrmQ detail page enhanced: now displaying args/kwargs/q_options instead of encrypted payload.

* if `cluster` argument is not set (the default), async_task() and schedule() will be handled by the default cluster; Documentation update.

* Text cleanup

* Documentation update.

* Fix TIMEOUT setting in Windows for non-default cluster

---------

Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2023-04-02 16:36:17 +02:00
GDay
86467dcf03 Revert "Merge pull request #15 from django-q2/error-on-timeout"
This reverts commit d97dca5bcb, reversing
changes made to 01cb652071.
2023-03-31 02:22:38 +02:00
Jim Jagielski
f5ecd337ad Merge branch 'master' into error-on-timeout 2023-02-17 12:33:47 -05:00
msabatier
17c1609f10 Add meaningfull process titles with currently running task name (#57)
* Customize process names (in ps/top) and add task name in logs

* Increase severity of reincarnate log messages and add task name

* Document setproctitle optional dependency

Co-authored-by: Marc Sabatier <37879561+msabatier@users.noreply.github.com>
Co-authored-by: Stan Triepels <1939656+GDay@users.noreply.github.com>
2023-01-26 02:21:56 +01:00
msabatier
650c3b1524 Fix use of database router for write queries and remove Conf.HAS_REPLICA (#61)
Co-authored-by: Marc Sabatier
2023-01-18 02:12:13 +01:00
msabatier
a013591de5 Add intended_date_kwarg field to Schedule (#62)
Co-authored-by: Marc Sabatier <marc.sabatier@eco-adapt.com>
2023-01-17 22:52:11 +01:00
msabatier
8cd1028391 More explicit log messages in exception handling (#59) 2023-01-11 00:36:35 +01:00
Stan Triepels
ce81059da8 Fix: allow both ZoneInfo and Pytz depending on django version (#55) 2022-12-21 03:42:38 +01:00
Stan Triepels
e78e473be3 Fix: handling exceptions inside job function (#51) 2022-12-21 01:44:59 +01:00
Stan Triepels
4d454e4001 Fix: Daylight saving time issue with scheduler (#47) 2022-12-21 01:44:34 +01:00
Andreas Andersen
2566955782 Log exceptions with logger.exception (#42)
Co-authored-by: Andreas Bok Andersen <andreas.bokandersen@glooko.com>
2022-11-30 15:03:20 +01:00
Stan Triepels
3c0c32758d Chore: Flake8, isort and Black (#40) 2022-11-16 01:45:40 +01:00
Stan Triepels
542e04db54 Feat: Add biweekly and bimonthly and fix translations (#36) 2022-11-13 01:00:11 +01:00
Stan Triepels
6f19b3d270 Fix: func reference in admin (#28) 2022-11-05 02:45:44 +01:00
Stan Triepels
0aac4e7b0d Fix unclear error when function is not called correctly (#19) 2022-10-20 17:24:17 +02:00
Stan Triepels
8822e7d409 Remove funding file, add docker/compose for development project and fix (#18)
task names in logs
2022-10-20 15:17:44 +02:00
Noortheen Raja
64d73a77b7 feat: option to fail-on-timeout
fixes #251
2022-10-19 02:27:12 +02:00
Stan Triepels
c59f036158 Remove arrow dependency, update packages, set up testing with compose (#17) 2022-10-19 02:01:16 +02:00
Stan Triepels
ce98aaa15c allow atomic on external db (#14)
Co-authored-by: Rajeev KL <rajeev@labmoneta.io>
2022-10-15 23:24:30 +02:00
Stan Triepels
9ace854290 Save limit per group/func/name (#12)
* feat: option to save tasks per group/func/name

* test: update tests for save limit

* fix: convert func when save-limit checking

when passed as function it needs to be converted to work

* Add warning if option is not valid

Co-authored-by: Noortheen Raja <jnoortheen@gmail.com>
2022-10-15 01:56:41 +02:00
Stan Triepels
e3f959c258 Admin improvements (#11)
* Add the group column to OrmQ admin page

* Add group field to tasks list and search_field

* Add search by name to Schedule admin page

* Limit succesful task history per schedule with SAVE_LIMIT_GROUP config

* Improve ScheduleAdmin list view performance

Co-authored-by: Marc Sabatier <marc@sabatier.online>
2022-10-14 17:13:48 +02:00
Stan Triepels
4aa6325ae8 Replace use of eval() by ast.parse() + ast.literal_eval() (#10)
Co-authored-by: Marc Sabatier <marc@sabatier.online>
2022-10-14 16:04:38 +02:00
Stan Triepels
803618da03 Fix connection issues with CONN_MAX_AGE > 0 (#9)
Co-authored-by: Marc Sabatier <marc@sabatier.online>
2022-10-14 02:08:48 +02:00
Stan Triepels
c4ebfc66b9 Show function name in log when running task (#3)
* refactor: show function name as it runs

* feat: log func name when processed

Co-authored-by: Noortheen Raja <jnoortheen@gmail.com>
2022-09-17 01:48:07 +02:00
Ilan Steemers
f8cd136668 Post execute signal and tests (#580)
* adds post_execute signal and tests

* Updates signal docts

* Fixes typo in docs
2021-06-20 15:43:08 +02:00
Ilan Steemers
4df036dc9c Autofield (#574)
* Adds autofield default

* linting

* Updates packages
2021-06-10 15:42:21 +02:00
Ilan Steemers
15155c7a99 Build improvements (#569)
* Black linting

* Adding Black to dev dependencies and upping minimal python to 3.6.2 for compatibility

* Updating packages

* Removing pip-tools input and exporting requirements with poetry

* Deleting old setup files and test runner

* Trying 1.3.7

* Looser extras requirements to prevent conflicts

* Sorted imports with isort

* Added iSort to dev dependencies

* Fixes localtime for naive setups
2021-05-30 17:10:07 +02:00
Alexandre Xavier
71d4b726ea Feature/improves multiple databases support (#561)
* feat: adds docker local files to ignore

* feat: adds replica key to django q settings

* feat: adds multiple database routers testing utility

* feat: ensures scheduler works with multiple databases and replicas

* feat: adds docker image to support containerized development

* refactor: removes unnecessary import

* style: improves import styling

* docs: adds replica setting to configure documentation
2021-05-23 15:10:31 +02:00
nittolese
8d49825d40 Fix #422 (#556) 2021-05-14 13:49:50 +02:00
Kenny Heinonen
1eb5cf4b9b Fixes #314 - Convert func to string before saving task in databse so that resubmitting failed task works (#554) 2021-05-14 13:48:08 +02:00
Dimitri
957f807598 Allow tasks to be scheduled on a specific cluster (#555)
* Allows schedule by cluster

* Update docs

* Add example in doc

* Fixes cluster field must be blank

* Adds cluster arg to schedule func

* Adds cluster field in list_display/filter for admin

* Uses Q filter to fetch schedule

* Fixes ref in doc

* Adds tests
2021-05-14 13:42:54 +02:00
Kenny Heinonen
ad77ad7512 Fixes #225 (#552) 2021-05-07 17:26:41 +02:00
weiyang
1869a94118 Use 'timezone.localtime()' when calculating the next run time (#520)
Signed-off-by: weiyang <weiyang.ones@gmail.com>
2021-03-21 12:02:22 +01:00
Ilan Steemers
7d03536e89 Migrate to Github Action CI (#507)
* Some small refactors

* Initial test script

* Fixes django matrix

* Trying to set up Disque

* Disque as a docker service

* lowercase action name

* Remove Travis

* Replaces Travis badge with Github actions badge

* Show django version in step

* Typo
2021-02-14 18:59:04 +01:00
Telmo Barros
1d6d996fd0 enhanced admin ormq panel fix and schedules with broker_name 2021-02-06 18:52:17 +00:00
ihuk
8b3d6b7d2a Initial work on fix for #424 - TypeError: can't pickle _thread.lock objects.
Made `Broker` and `Queue` classes pickable which fixes the the problem with new default `spawn` proces start method.
2020-10-14 22:21:40 +02:00
Tim O'Meara
655f0aadb5 rename config var to max_retries, remove attempt_count form admin , add admin ui customization to docs 2020-08-13 11:58:00 -05:00
Tim O'Meara
45b0b2cc27 add attempt_count to config docs, set value to 1 in inital task save 2020-08-11 10:42:28 -05:00
Tim O'Meara
3c3395f24b add attempt_count to task 2020-08-11 10:16:31 -05:00
Ilan Steemers
c94f2cb816 Adds psutil support for max_rss on other platforms 2020-07-05 18:18:43 +02:00
Ilan Steemers
b16878f36f Allow for platforms not supporting resource module 2020-07-05 17:45:07 +02:00
Ilan Steemers
83b8b7c0d7 Adds max_rss memory limit for recycle 2020-07-05 17:04:16 +02:00
Ilan Steemers
f0061dccd6 Moves optional import to root 2020-06-29 19:08:33 +02:00
Ilan Steemers
5bf2cd49eb Fixes deprecation 2020-06-28 20:23:46 +02:00
Ilan Steemers
ace3d01da7 Optional Cron expression scheduling - inital commit 2020-06-28 15:26:53 +02:00
Ilan Steemers
6e95dc2d9b Hints and lints 2020-06-15 16:02:14 +02:00
Ilan Steemers
ef4c7876ae Linting and hints 2020-06-12 12:33:04 +02:00
Bastien Vallet
68d95a2924 [cleanup] Remove python2 artifacts 2020-06-10 11:48:31 +02:00
Ilan Steemers
a6eda43c72 Merge pull request #428 from hhyo/hhyo-patch-1
Fix issue when using USE_TZ=False with MySQL
2020-05-27 17:49:38 +02:00