Commit Graph
145 Commits
Author SHA1 Message Date
Stan TriepelsandRajeev KL ce98aaa15c allow atomic on external db (#14)
Co-authored-by: Rajeev KL <rajeev@labmoneta.io>
2022-10-15 23:24:30 +02:00
Stan TriepelsandNoortheen Raja 9ace854290 Save limit per group/func/name (#12)
* feat: option to save tasks per group/func/name

* test: update tests for save limit

* fix: convert func when save-limit checking

when passed as function it needs to be converted to work

* Add warning if option is not valid

Co-authored-by: Noortheen Raja <jnoortheen@gmail.com>
2022-10-15 01:56:41 +02:00
Stan TriepelsandMarc Sabatier e3f959c258 Admin improvements (#11)
* Add the group column to OrmQ admin page

* Add group field to tasks list and search_field

* Add search by name to Schedule admin page

* Limit succesful task history per schedule with SAVE_LIMIT_GROUP config

* Improve ScheduleAdmin list view performance

Co-authored-by: Marc Sabatier <marc@sabatier.online>
2022-10-14 17:13:48 +02:00
Stan TriepelsandMarc Sabatier 4aa6325ae8 Replace use of eval() by ast.parse() + ast.literal_eval() (#10)
Co-authored-by: Marc Sabatier <marc@sabatier.online>
2022-10-14 16:04:38 +02:00
Stan TriepelsandMarc Sabatier 803618da03 Fix connection issues with CONN_MAX_AGE > 0 (#9)
Co-authored-by: Marc Sabatier <marc@sabatier.online>
2022-10-14 02:08:48 +02:00
Stan TriepelsandNoortheen Raja c4ebfc66b9 Show function name in log when running task (#3)
* refactor: show function name as it runs

* feat: log func name when processed

Co-authored-by: Noortheen Raja <jnoortheen@gmail.com>
2022-09-17 01:48:07 +02:00
Ilan Steemers f8cd136668 Post execute signal and tests (#580)
* adds post_execute signal and tests

* Updates signal docts

* Fixes typo in docs
2021-06-20 15:43:08 +02:00
Ilan Steemers 4df036dc9c Autofield (#574)
* Adds autofield default

* linting

* Updates packages
2021-06-10 15:42:21 +02:00
Ilan Steemers 15155c7a99 Build improvements (#569)
* Black linting

* Adding Black to dev dependencies and upping minimal python to 3.6.2 for compatibility

* Updating packages

* Removing pip-tools input and exporting requirements with poetry

* Deleting old setup files and test runner

* Trying 1.3.7

* Looser extras requirements to prevent conflicts

* Sorted imports with isort

* Added iSort to dev dependencies

* Fixes localtime for naive setups
2021-05-30 17:10:07 +02:00
Alexandre Xavier 71d4b726ea Feature/improves multiple databases support (#561)
* feat: adds docker local files to ignore

* feat: adds replica key to django q settings

* feat: adds multiple database routers testing utility

* feat: ensures scheduler works with multiple databases and replicas

* feat: adds docker image to support containerized development

* refactor: removes unnecessary import

* style: improves import styling

* docs: adds replica setting to configure documentation
2021-05-23 15:10:31 +02:00
nittolese 8d49825d40 Fix #422 (#556) 2021-05-14 13:49:50 +02:00
Kenny Heinonen 1eb5cf4b9b Fixes #314 - Convert func to string before saving task in databse so that resubmitting failed task works (#554) 2021-05-14 13:48:08 +02:00
Dimitri 957f807598 Allow tasks to be scheduled on a specific cluster (#555)
* Allows schedule by cluster

* Update docs

* Add example in doc

* Fixes cluster field must be blank

* Adds cluster arg to schedule func

* Adds cluster field in list_display/filter for admin

* Uses Q filter to fetch schedule

* Fixes ref in doc

* Adds tests
2021-05-14 13:42:54 +02:00
Kenny Heinonen ad77ad7512 Fixes #225 (#552) 2021-05-07 17:26:41 +02:00
weiyang 1869a94118 Use 'timezone.localtime()' when calculating the next run time (#520)
Signed-off-by: weiyang <weiyang.ones@gmail.com>
2021-03-21 12:02:22 +01:00
Ilan Steemers 7d03536e89 Migrate to Github Action CI (#507)
* Some small refactors

* Initial test script

* Fixes django matrix

* Trying to set up Disque

* Disque as a docker service

* lowercase action name

* Remove Travis

* Replaces Travis badge with Github actions badge

* Show django version in step

* Typo
2021-02-14 18:59:04 +01:00
Telmo Barros 1d6d996fd0 enhanced admin ormq panel fix and schedules with broker_name 2021-02-06 18:52:17 +00:00
ihuk 8b3d6b7d2a Initial work on fix for #424 - TypeError: can't pickle _thread.lock objects.
Made `Broker` and `Queue` classes pickable which fixes the the problem with new default `spawn` proces start method.
2020-10-14 22:21:40 +02:00
Tim O'Meara 655f0aadb5 rename config var to max_retries, remove attempt_count form admin , add admin ui customization to docs 2020-08-13 11:58:00 -05:00
Tim O'Meara 45b0b2cc27 add attempt_count to config docs, set value to 1 in inital task save 2020-08-11 10:42:28 -05:00
Tim O'Meara 3c3395f24b add attempt_count to task 2020-08-11 10:16:31 -05:00
Ilan Steemers c94f2cb816 Adds psutil support for max_rss on other platforms 2020-07-05 18:18:43 +02:00
Ilan Steemers b16878f36f Allow for platforms not supporting resource module 2020-07-05 17:45:07 +02:00
Ilan Steemers 83b8b7c0d7 Adds max_rss memory limit for recycle 2020-07-05 17:04:16 +02:00
Ilan Steemers f0061dccd6 Moves optional import to root 2020-06-29 19:08:33 +02:00
Ilan Steemers 5bf2cd49eb Fixes deprecation 2020-06-28 20:23:46 +02:00
Ilan Steemers ace3d01da7 Optional Cron expression scheduling - inital commit 2020-06-28 15:26:53 +02:00
Ilan Steemers 6e95dc2d9b Hints and lints 2020-06-15 16:02:14 +02:00
Ilan Steemers ef4c7876ae Linting and hints 2020-06-12 12:33:04 +02:00
Bastien Vallet 68d95a2924 [cleanup] Remove python2 artifacts 2020-06-10 11:48:31 +02:00
Ilan Steemers a6eda43c72 Merge pull request #428 from hhyo/hhyo-patch-1
Fix issue when using USE_TZ=False with MySQL
2020-05-27 17:49:38 +02:00
Ilan Steemers bef944af61 Merge pull request #440 from maerteijn/fix-database-router
Fix issues when using multiple databases with a database router
2020-05-27 17:48:20 +02:00
Martijn Jacobs f0546bc91c Lock the Scheduler model for the database it's using
This can happen when you use a database router for example
2020-05-01 17:30:19 +02:00
小圈圈 25d1e3e3a0 Fix issue when using USE_TZ=False with MySQL #353 2020-04-15 20:57:27 +08:00
Ryan Branche 2f14c122e4 When sync=True, re-raise exceptions from the worker.
This commit adds code to detect if the task is running synchronously in
the worker and re-raise exceptions that were originally raised by the
underlying task function.
2020-02-28 14:26:20 -08:00
Ilan 0bca3cbd7a Replacing format with f-strings 2020-02-17 16:59:12 +01:00
Jason McVetta 4529c38067 log cluster_id instead of pid 2020-02-12 18:26:53 +07:00
Jason McVetta 14a602ee8b use both pid and cluster_id 2020-02-12 18:23:47 +07:00
Jason McVetta d22c916bfc add cluster_id to Sentinel 2020-02-12 17:53:52 +07:00
Jason McVetta ea30c1d43f add UUID4-based cluster_id 2020-02-12 17:41:10 +07:00
Ilan Steemers 681375364d Merge pull request #393 from ossobv/sync
Preserve database connection when sync=True
2020-01-17 15:34:46 +01:00
Jake Bell dc65f5e1de Replacing ugettext_ functions with gettext_ for Django 3 2020-01-07 15:57:37 -06:00
Harm Geerts 66588a28e7 Preserve database connection when sync=True
When tasks run synchronously django-q should only affect the calling
context by running the tasks.
Fixes https://github.com/Koed00/django-q/issues/326
2019-11-27 12:37:31 +01:00
Ilan Steemers c27938ee62 Merge pull request #333 from tremby/fix-timeout-override
Fix timeout override
2019-08-13 14:16:03 +02:00
Martijn Jacobs 924ff928b0 Lock the Schedule objects to ensure tasks are created once
Tested this with PostgreSQL and MySQL
2019-08-12 14:28:26 +02:00
Martijn Jacobs abaa0ba39b Don't call the scheduler twice: respect Conf.SCHEDULER 2019-08-12 14:27:56 +02:00
Ilan Steemers 7d437ea5cf Merge pull request #353 from maerteijn/fix-mysql-naive-datetime-in-scheduler
Fix issue when using USE_TZ=False with MySQL
2019-08-10 14:16:11 +02:00
Ilan Steemers 16fa6fa8d9 Fixes deprecated arrow syntax 2019-08-10 14:03:03 +02:00
Martijn Jacobs c0cda97864 Don't save the next_run field as TZ aware when USE_TZ is False
arrow always returns a tz aware datetime, and we don't want this when we explicitly configured django with USE_TZ=False. This causes issues with MySQL.
2019-04-17 15:02:20 +02:00
Janne Rönkkö bca22054df Fix concurrency issue in timeout timer value processing
According to multiprocessing documentation for Value
(https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Value)
reads and writes are protected with lock when the lock argument is True
(the default) or the lock argument is an instance of Lock or RLock. The
documentation states that operations like += are not atomic as that
involves reading and writing.

On the worker side the critical section includes also storing finished task
result because the timeout could happen after the task function has
finished but before the result has been stored and timer.value has been
updated to tell the guard process that the task has been finished.

On the guard side the critical section includes all checks done to see
if the worker has timed out or died and the actual reincarnation function
because the worker could update timer value to -1 (idle) or -2 (recycle)
after the guard has seen timer value 0 (timeout) and is going to terminate
the worker.
2019-01-28 08:39:56 +02:00