Commit Graph
135 Commits
Author SHA1 Message Date
nittolese 8d49825d40 Fix #422 (#556) 2021-05-14 13:49:50 +02:00
Kenny Heinonen 1eb5cf4b9b Fixes #314 - Convert func to string before saving task in databse so that resubmitting failed task works (#554) 2021-05-14 13:48:08 +02:00
Dimitri 957f807598 Allow tasks to be scheduled on a specific cluster (#555)
* Allows schedule by cluster

* Update docs

* Add example in doc

* Fixes cluster field must be blank

* Adds cluster arg to schedule func

* Adds cluster field in list_display/filter for admin

* Uses Q filter to fetch schedule

* Fixes ref in doc

* Adds tests
2021-05-14 13:42:54 +02:00
Kenny Heinonen ad77ad7512 Fixes #225 (#552) 2021-05-07 17:26:41 +02:00
weiyang 1869a94118 Use 'timezone.localtime()' when calculating the next run time (#520)
Signed-off-by: weiyang <weiyang.ones@gmail.com>
2021-03-21 12:02:22 +01:00
Ilan Steemers 7d03536e89 Migrate to Github Action CI (#507)
* Some small refactors

* Initial test script

* Fixes django matrix

* Trying to set up Disque

* Disque as a docker service

* lowercase action name

* Remove Travis

* Replaces Travis badge with Github actions badge

* Show django version in step

* Typo
2021-02-14 18:59:04 +01:00
Telmo Barros 1d6d996fd0 enhanced admin ormq panel fix and schedules with broker_name 2021-02-06 18:52:17 +00:00
ihuk 8b3d6b7d2a Initial work on fix for #424 - TypeError: can't pickle _thread.lock objects.
Made `Broker` and `Queue` classes pickable which fixes the the problem with new default `spawn` proces start method.
2020-10-14 22:21:40 +02:00
Tim O'Meara 655f0aadb5 rename config var to max_retries, remove attempt_count form admin , add admin ui customization to docs 2020-08-13 11:58:00 -05:00
Tim O'Meara 45b0b2cc27 add attempt_count to config docs, set value to 1 in inital task save 2020-08-11 10:42:28 -05:00
Tim O'Meara 3c3395f24b add attempt_count to task 2020-08-11 10:16:31 -05:00
Ilan Steemers c94f2cb816 Adds psutil support for max_rss on other platforms 2020-07-05 18:18:43 +02:00
Ilan Steemers b16878f36f Allow for platforms not supporting resource module 2020-07-05 17:45:07 +02:00
Ilan Steemers 83b8b7c0d7 Adds max_rss memory limit for recycle 2020-07-05 17:04:16 +02:00
Ilan Steemers f0061dccd6 Moves optional import to root 2020-06-29 19:08:33 +02:00
Ilan Steemers 5bf2cd49eb Fixes deprecation 2020-06-28 20:23:46 +02:00
Ilan Steemers ace3d01da7 Optional Cron expression scheduling - inital commit 2020-06-28 15:26:53 +02:00
Ilan Steemers 6e95dc2d9b Hints and lints 2020-06-15 16:02:14 +02:00
Ilan Steemers ef4c7876ae Linting and hints 2020-06-12 12:33:04 +02:00
Bastien Vallet 68d95a2924 [cleanup] Remove python2 artifacts 2020-06-10 11:48:31 +02:00
Ilan Steemers a6eda43c72 Merge pull request #428 from hhyo/hhyo-patch-1
Fix issue when using USE_TZ=False with MySQL
2020-05-27 17:49:38 +02:00
Ilan Steemers bef944af61 Merge pull request #440 from maerteijn/fix-database-router
Fix issues when using multiple databases with a database router
2020-05-27 17:48:20 +02:00
Martijn Jacobs f0546bc91c Lock the Scheduler model for the database it's using
This can happen when you use a database router for example
2020-05-01 17:30:19 +02:00
小圈圈 25d1e3e3a0 Fix issue when using USE_TZ=False with MySQL #353 2020-04-15 20:57:27 +08:00
Ryan Branche 2f14c122e4 When sync=True, re-raise exceptions from the worker.
This commit adds code to detect if the task is running synchronously in
the worker and re-raise exceptions that were originally raised by the
underlying task function.
2020-02-28 14:26:20 -08:00
Ilan 0bca3cbd7a Replacing format with f-strings 2020-02-17 16:59:12 +01:00
Jason McVetta 4529c38067 log cluster_id instead of pid 2020-02-12 18:26:53 +07:00
Jason McVetta 14a602ee8b use both pid and cluster_id 2020-02-12 18:23:47 +07:00
Jason McVetta d22c916bfc add cluster_id to Sentinel 2020-02-12 17:53:52 +07:00
Jason McVetta ea30c1d43f add UUID4-based cluster_id 2020-02-12 17:41:10 +07:00
Ilan Steemers 681375364d Merge pull request #393 from ossobv/sync
Preserve database connection when sync=True
2020-01-17 15:34:46 +01:00
Jake Bell dc65f5e1de Replacing ugettext_ functions with gettext_ for Django 3 2020-01-07 15:57:37 -06:00
Harm Geerts 66588a28e7 Preserve database connection when sync=True
When tasks run synchronously django-q should only affect the calling
context by running the tasks.
Fixes https://github.com/Koed00/django-q/issues/326
2019-11-27 12:37:31 +01:00
Ilan Steemers c27938ee62 Merge pull request #333 from tremby/fix-timeout-override
Fix timeout override
2019-08-13 14:16:03 +02:00
Martijn Jacobs 924ff928b0 Lock the Schedule objects to ensure tasks are created once
Tested this with PostgreSQL and MySQL
2019-08-12 14:28:26 +02:00
Martijn Jacobs abaa0ba39b Don't call the scheduler twice: respect Conf.SCHEDULER 2019-08-12 14:27:56 +02:00
Ilan Steemers 7d437ea5cf Merge pull request #353 from maerteijn/fix-mysql-naive-datetime-in-scheduler
Fix issue when using USE_TZ=False with MySQL
2019-08-10 14:16:11 +02:00
Ilan Steemers 16fa6fa8d9 Fixes deprecated arrow syntax 2019-08-10 14:03:03 +02:00
Martijn Jacobs c0cda97864 Don't save the next_run field as TZ aware when USE_TZ is False
arrow always returns a tz aware datetime, and we don't want this when we explicitly configured django with USE_TZ=False. This causes issues with MySQL.
2019-04-17 15:02:20 +02:00
Janne Rönkkö bca22054df Fix concurrency issue in timeout timer value processing
According to multiprocessing documentation for Value
(https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Value)
reads and writes are protected with lock when the lock argument is True
(the default) or the lock argument is an instance of Lock or RLock. The
documentation states that operations like += are not atomic as that
involves reading and writing.

On the worker side the critical section includes also storing finished task
result because the timeout could happen after the task function has
finished but before the result has been stored and timer.value has been
updated to tell the guard process that the task has been finished.

On the guard side the critical section includes all checks done to see
if the worker has timed out or died and the actual reincarnation function
because the worker could update timer value to -1 (idle) or -2 (recycle)
after the guard has seen timer value 0 (timeout) and is going to terminate
the worker.
2019-01-28 08:39:56 +02:00
Ilan Steemers 44d963b716 Merge pull request #327 from mattaw/master
Modified django_q imports to support Python 3.4 again in cluster.py. …
2019-01-27 18:34:30 +01:00
Ilan Steemers 198ab59555 Merge branch 'master' into fix-timeout-override 2019-01-27 18:30:05 +01:00
Janne Rönkkö 0e2df88d92 Fix async_task timeout parameter handling when cluster timeout is set to None (the default)
The cluster timeout configuration default value None is documented to mean
tasks never timeout out. Also the documentation states that the timeout
can be overridden for individual tasks.

With the old implementation the timeout parameter given to async_task was
not honored if the cluster timeout was set to None.

Fixes: #335
2019-01-27 19:09:22 +02:00
Bart Nagel 4c2a2f3279 Fix timeout override
Closes https://github.com/Koed00/django-q/issues/332
2019-01-21 16:23:45 -08:00
Matthew Swabey e84be5f33b Modified django_q imports to support Python 3.4 again in cluster.py. Closes #315 2018-12-09 12:28:04 -05:00
Ilan Steemers b5a01a9e93 Removes old rollbar support - relies on the pluggable error reporters now 2018-08-10 13:03:58 +02:00
Ilan Steemers 7d5c9f8b46 removes xrange compat 2018-08-05 13:39:04 +02:00
Ilan Steemers bd062503c5 Adds tracebacks in error logs @strets123 2018-08-05 11:13:16 +02:00
Ilan Steemers acebdaa850 Renames async/enqueue to async_task. 2018-08-01 15:20:34 +02:00
Pierre-Elliott Bécue ae75a84f7f Replaces async occurrences with alternatives
* async is now a reserved word in python3.7
 * Rename async function to enqueue
 * Rename all async_ functions to enqueue_
 * Rename Async class to AsyncTask
 * Updates the docs.
2018-07-07 21:33:43 +02:00